-- Logs begin at Sat 2022-09-10 20:21:31 UTC, end at Sat 2022-09-10 21:20:32 UTC. -- Sep 10 20:45:20.547697 dv-cinder-851996-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 10 20:45:20.554850 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 10 20:45:20 2022] *** Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: nodename: dv-cinder-851996-9-iscsi Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: machine: x86_64 Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: clock source: unix Sep 10 20:45:20.556840 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: pcre jit disabled Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: detected number of CPU cores: 4 Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: current working directory: / Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: detected binary path: /usr/bin/uwsgi-core Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: your processes number limit is 47680 Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: your memory page size is 4096 bytes Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: detected max file descriptor number: 2048 Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: lock engine: pthread robust mutexes Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: thunder lock: enabled Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 10 20:45:20.557301 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 10 20:45:20.598131 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: Python main interpreter initialized at 0x555b874769c0 Sep 10 20:45:20.598131 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: python threads support enabled Sep 10 20:45:20.598131 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: your server socket listen backlog is limited to 100 connections Sep 10 20:45:20.598131 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: your mercy for graceful operations on workers is 90 seconds Sep 10 20:45:20.598541 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: mapped 403077 bytes (393 KB) for 2 cores Sep 10 20:45:20.598623 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: *** Operational MODE: preforking *** Sep 10 20:45:20.598864 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: *** uWSGI is running in multiple interpreter mode *** Sep 10 20:45:20.598864 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: spawned uWSGI master process (pid: 81426) Sep 10 20:45:20.599068 dv-cinder-851996-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 10 20:45:20.600713 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: spawned uWSGI worker 1 (pid: 81440, cores: 1) Sep 10 20:45:20.600713 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: spawned uWSGI worker 2 (pid: 81441, cores: 1) Sep 10 20:45:20.600713 dv-cinder-851996-9-iscsi devstack@keystone.service[81426]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 10 20:45:24.914172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:45:24.914172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: from cryptography.utils import int_from_bytes Sep 10 20:45:24.914837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:45:24.914837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: from cryptography.utils import int_from_bytes Sep 10 20:45:24.915276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:45:24.915276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: from cryptography.utils import int_from_bytes Sep 10 20:45:24.915425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:45:24.915425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: from cryptography.utils import int_from_bytes Sep 10 20:45:24.964381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 20:45:24.964567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 20:45:24.964707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 20:45:24.964799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] command line args: [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 20:45:24.964904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 20:45:24.964983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 20:45:24.965098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] command line args: [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 20:45:24.965184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ================================================================================ {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 20:45:24.965373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 20:45:24.965479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] admin_token = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.965590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ================================================================================ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 20:45:24.965678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] config_dir = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.965827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.965904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] admin_token = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] config_source = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config_dir = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] debug = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config_source = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] debug = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.966990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_config_append = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.967990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.968099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.968204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_config_append = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.968298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_dir = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.968873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.968976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_file = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_dir = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.969982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.970986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.971962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] publish_errors = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.972974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] publish_errors = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.973904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] standard_threads = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] standard_threads = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.974960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] use_journal = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] use_json = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] use_stderr = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] use_syslog = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_journal = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.975783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_json = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.976002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.976122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_stderr = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.976232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.976341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_syslog = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.976457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.976573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:45:24.976697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.976808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.976919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.977950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.978981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.979907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.980887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.981951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.982965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.983950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.984672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.984801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.984914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.985969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.986946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.987982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.988964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.989760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.990935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.991977 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.992897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.993971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.994992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.995889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.996968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.997956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.998898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:24.999838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.000984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.external = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.001932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.password = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] auth.token = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.external = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.002847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.password = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.token = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.003970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.004892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.005936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.006970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.007994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.008921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.009982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.010901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.011981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.012901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.013983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.014904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.015899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.016921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.017886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.018937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.019957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.020984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.021921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.022831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.023941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.024991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.025979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.026922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.027947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.028964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.029901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.030790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.031395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.031536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.031699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.031816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.031929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.032948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.033898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] role.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] role.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.034981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.035930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.036954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.037960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.038998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.039916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.040922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.041876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.042908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.043970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.044900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.045965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.connection = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.046928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.047973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.048988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.049924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 20:45:25.050713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.050713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.051327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.051327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.051327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.051645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:45:25.051645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 20:45:25.079411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.079632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.079856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.081159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.081394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.081670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.088709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.090179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.118253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.118417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.118505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.118583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.175056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.175800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.223944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.224176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.224271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.224410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.381705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.381926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.382891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.383017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.383240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.407975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.408070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.408194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.408278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.408410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.408553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.408855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.419783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.420360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.432894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.433981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.434064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.434195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.434286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:45:25.451615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.451825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:45:25.451911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.452047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:45:25.452175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.452380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:45:25.452464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.452577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:45:25.452659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.453401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.453549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.453719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.453803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.453981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.454117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.454239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.463636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.463923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.464200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.464488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.467217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.467609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.467876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.468150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.472186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.475127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.475418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.475711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.475794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.476001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.476295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.476565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.476844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.477127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.477406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.477711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.477988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.478710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.478993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.479273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.479554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.479839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.480099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.480365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.480637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.480905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.481177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.481444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.501797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.503285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.503577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.504880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.506385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.506669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.508304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.508594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.508883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.511355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.511641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.511923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.516222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.516509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.519336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.519621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.522268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.522574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.522871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.523170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.523459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.523747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.524029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.524309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.525294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.525613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.525899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.526182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.526475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.526732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.527000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.527266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.540012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.540298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.543069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.543345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.544203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.547565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.547667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.547912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.551026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.551325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.552179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.555408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.555549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.558699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.558789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.559003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.559293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.559584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.559870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.560174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.562067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.562941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.563229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.563506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.563793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.564070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.576681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.576978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.577254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.577550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.580121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.580403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.580668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.580929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.584619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.586203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.587738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.587871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.588022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.589351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.590812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.590922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.591104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.591188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.591397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.591685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.592080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.594388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.594669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.594954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.595234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.595615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.604027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.604321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.604610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.604898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.605283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.605610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.605898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.606918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.607200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.607479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.607759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.608135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.608422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.608700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.621636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.624291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.624586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.628142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.628231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.629690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.631099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.631184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.631460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.631857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.632613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.634117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.634398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.634791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.638585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.640058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.641811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.643070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.643267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.643469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.643762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.644045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.646206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.646613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.647030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.647366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.652041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.652346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.652633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.655285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.655585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.656382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.660233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.660522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.660810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.661094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.661377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.661681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.661968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.662260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.662548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.662834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.663119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.663393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.663507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:45:25.663664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.663786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.663929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.664064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.664336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.664609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.664879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.665149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.665429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.665734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.666011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.666289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.666555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.666817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.667074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:45:25.699441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555b874769c0 pid: 81441 (default app) Sep 10 20:45:25.701785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555b874769c0 pid: 81440 (default app) Sep 10 20:45:25.702701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea54f91-26e4-450b-85ff-d5ae870a8980 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:25.703397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea54f91-26e4-450b-85ff-d5ae870a8980 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:25.703568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea54f91-26e4-450b-85ff-d5ae870a8980 None None] PATH_INFO: `/v3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:25.707643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Sat Sep 10 20:45:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:32.509559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:32.510169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:32.510169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:32.514604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:45:32.520211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:32.520729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:32.520828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:32.590745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: INFO dbcounter [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] Registered counter for database keystone Sep 10 20:45:32.610897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG oslo_db.sqlalchemy.engines [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 20:45:32.617008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writer thread running {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 20:45:32.796815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG passlib.handlers.bcrypt [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81441) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 20:45:32.797204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG passlib.handlers.bcrypt [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81441) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 20:45:32.854869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:32.856447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:32.982485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 10 20:45:33.426282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:33.495843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: INFO dbcounter [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] Registered counter for database keystone Sep 10 20:45:33.517614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG oslo_db.sqlalchemy.engines [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81440) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 20:45:33.523655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writer thread running {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 20:45:33.643502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02d041-a893-4aed-b37b-0f76bbd0414b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:33.647738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:33.648368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:33.648652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:33.648927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:33.657876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:33.710972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 20:45:33.711259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce680a0-ddbb-4d24-96da-db0518c67add admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:33.721904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Sat Sep 10 20:45:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:45:33.727845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:33.744475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320f6f-9cec-49e0-9758-3f9fa99cf3cf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:33.748829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:33.750561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:33.750849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:33.751135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] PATH_INFO: `/v3/projects/51e33b7d1c8d453f9d66204642c03a87` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:33.752735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] RBAC: Authorizing `identity:get_project(project_id=51e33b7d1c8d453f9d66204642c03a87)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:33.805642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING keystone.common.rbac_enforcer.enforcer [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 20:45:33.805993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543f1f-f3c9-45e6-9264-9883c3df5714 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:33.810455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Sat Sep 10 20:45:33 2022] GET /identity/v3/projects/51e33b7d1c8d453f9d66204642c03a87 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:35.923241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217c59-3df9-455a-a5e3-feb8c70ec022 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:35.923653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217c59-3df9-455a-a5e3-feb8c70ec022 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:35.923795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217c59-3df9-455a-a5e3-feb8c70ec022 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:35.927907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:35.933386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:35.933777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:35.933973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:35.955112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:35.956639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:35.964698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 10 20:45:36.236986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2217c59-3df9-455a-a5e3-feb8c70ec022 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:36.248939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2217c59-3df9-455a-a5e3-feb8c70ec022 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:36.253797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3c418-828c-4518-a062-1018d4d00dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:36.254396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3c418-828c-4518-a062-1018d4d00dac admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:36.254679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3c418-828c-4518-a062-1018d4d00dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:36.254952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3c418-828c-4518-a062-1018d4d00dac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:36.262512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3c418-828c-4518-a062-1018d4d00dac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:36.264551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3c418-828c-4518-a062-1018d4d00dac admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:36.274094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Sat Sep 10 20:45:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:45:36.279493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:36.288893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341fa3-0fd3-447b-bbac-1abce8d9d107 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:36.293148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1312a-87e2-4f1a-a159-8f6e771e7b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:36.293808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1312a-87e2-4f1a-a159-8f6e771e7b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:36.294092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1312a-87e2-4f1a-a159-8f6e771e7b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:36.294365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1312a-87e2-4f1a-a159-8f6e771e7b6a admin admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:36.295677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1312a-87e2-4f1a-a159-8f6e771e7b6a admin admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:36.298075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1312a-87e2-4f1a-a159-8f6e771e7b6a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:36.299592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Sat Sep 10 20:45:36 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:38.521074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64adea-ea55-4d8d-8560-5f2aebbb4b07 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:38.521628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64adea-ea55-4d8d-8560-5f2aebbb4b07 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:38.521739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64adea-ea55-4d8d-8560-5f2aebbb4b07 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:38.522990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:45:38.528397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:38.528786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:38.528944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:38.547500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:38.548671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:38.567325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cb607-ee9f-4670-ae59-92fbb14e41f4 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:38.567619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cb607-ee9f-4670-ae59-92fbb14e41f4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:38.567899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cb607-ee9f-4670-ae59-92fbb14e41f4 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:38.569249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:45:38.574506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:38.574798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:38.575079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:38.617468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG passlib.handlers.bcrypt [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81440) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 20:45:38.617884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG passlib.handlers.bcrypt [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81440) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 20:45:38.631818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:38.667571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:38.669151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:38.767516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:45:38.898544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:38.924292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.004856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30458e-bd0a-4ff1-8eb8-4e3a1bf330f6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.038665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.038865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.039297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.039577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.039851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.051081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:39.059835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:39.060922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:45:39.062191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:45:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:39.067297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.140009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46f77d-cecc-4a76-b494-f9e428d79555 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.142523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.149782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.169409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb55be-9696-4399-9b7c-328e61dda46c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.179606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.180481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.180762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.181031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.203513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.204125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.204408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.204682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.205347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:39.205683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.207083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.219177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:39.225054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.248491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.250097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.323613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403bf1c-4d55-40ec-b964-bdfd01644094 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.348765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:45:39.354286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.357377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.357801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.358050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.358332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.358608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.360442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.361999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.363562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:39.368822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.385388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.455762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5eb66-b0e0-4c39-9305-6a08745e5f5c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.470946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95e96-6fad-419b-9d75-8ec1b3af3d44 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.489525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.490393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.490677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.490955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.504078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.504709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.504985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.505253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.506391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.507841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.509357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:39.515215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.515392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.516702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:45:39.534998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:39.536304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:45:39.537093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:45:39.537093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:45:39.537093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:45:39.537093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:45:39.541819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:39.597174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.598892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:39.618302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea146119-b9d0-4a48-a09d-f06b65fa4206 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.645021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32456b-f119-4bdd-9f0f-3b57056bb05c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:39.652103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.652707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.652990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.653263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.654009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.domain.id=default, user.id=5da7578088e04700b6983b884a95e6ef) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:39.654310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.655741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f96e-e31b-4245-8fb0-4024e0334627 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.673491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/role_assignments?user.id=5da7578088e04700b6983b884a95e6ef&scope.domain.id=default&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:39.678516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:39.679161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:39.679442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:39.679718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:39.680390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:39.680697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:39.682172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe42aab-1880-495a-a6f4-5a00d25ae84d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:39.703574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:42.555387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4a86-9beb-46cb-a0f5-6a5b05cc1bdb None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.555897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4a86-9beb-46cb-a0f5-6a5b05cc1bdb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.555968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4a86-9beb-46cb-a0f5-6a5b05cc1bdb None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.557269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:42.564075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881a79-b133-46bf-8650-ab6e343cf4f5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.564611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881a79-b133-46bf-8650-ab6e343cf4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.564611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881a79-b133-46bf-8650-ab6e343cf4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.602667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9e881a79-b133-46bf-8650-ab6e343cf4f5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:42.603945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9e881a79-b133-46bf-8650-ab6e343cf4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:42.707050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:45:42.737140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3105fcd-6edb-48d6-a5a1-2faa21795502 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.737497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3105fcd-6edb-48d6-a5a1-2faa21795502 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.737721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3105fcd-6edb-48d6-a5a1-2faa21795502 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.739074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:42.759229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7686bb-1db7-4cd8-bc9f-d15d180e5a9f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.759531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7686bb-1db7-4cd8-bc9f-d15d180e5a9f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.759797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7686bb-1db7-4cd8-bc9f-d15d180e5a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.772284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e724b-37e5-4766-9a32-d091fbf9c1a8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.772576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e724b-37e5-4766-9a32-d091fbf9c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.773084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e724b-37e5-4766-9a32-d091fbf9c1a8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.774891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 14/25] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:42.777446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e22d5-0aad-4d54-84fc-7857bb95c560 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.777750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e22d5-0aad-4d54-84fc-7857bb95c560 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.778141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e22d5-0aad-4d54-84fc-7857bb95c560 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.780664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 15/26] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:42.782367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.782646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.782919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.788394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2c7686bb-1db7-4cd8-bc9f-d15d180e5a9f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:42.789615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2c7686bb-1db7-4cd8-bc9f-d15d180e5a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:42.817265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:42.818424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:42.854708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:42.880971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 12/27] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:42.883814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.883814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.883814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.932731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:42.933980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:42.983688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 16/28] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:43.066379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.071156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 13/29] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:43.176609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:43.204971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d978e-87d3-4764-9c7f-98680a1b4710 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:43.236771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.250626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:43.251493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:43.251911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:43.252322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:43.271561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:43.273609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:45:43.301891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:43.302771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:45:43.303575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:45:43.303575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:45:43.303575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:45:43.305620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 17/30] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:45:43.311729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.461775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90072-ea26-44a4-b8b8-6aaf3bd0940f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:43.479858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:43.499640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772a445-eb72-4a3b-b1f9-8b62367bf2d1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:43.511048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:43.512391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:43.512880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:43.513600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:43.528084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:43.530613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:43.537076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:43.538124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:45:43.538870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 14/31] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:45:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:43.539667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.553497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:43.553497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:43.553497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:43.553497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:43.553901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:43.553901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:43.553901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:43.572163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 18/32] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:43.573003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.684752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101c4a-ee2e-41d7-b623-2fe13f07e986 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:43.697595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f79f-82ee-4434-89ef-3528f27f1743 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:43.731018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:43.731563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:43.731847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:43.732123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:43.732879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:43.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:43.739983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:43.740887 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:43.741336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:43.741803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:43.743237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:43.744946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:43.879836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 19/33] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Sep 10 20:45:43 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:43.881254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.894919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 15/34] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Sep 10 20:45:43 2022] POST /identity/v3/roles => generated 219 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:43.895748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:43.949901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:43.987430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:44.022567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64411c-fadc-462e-8a80-c6688dc77dbb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.084402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:44.086173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.086696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.087147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.088115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:44.088628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:44.090457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf08aa5-40cf-4806-93e6-b90d9cbdbd9f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:44.110099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 20/35] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:44.110099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308723-bd1e-45f8-8805-7e4a4e9c444a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.110099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308723-bd1e-45f8-8805-7e4a4e9c444a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.110099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308723-bd1e-45f8-8805-7e4a4e9c444a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.111278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 21/36] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:44.112247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42914ce-0e3a-4ba4-92fb-98052fa56f1d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.112535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42914ce-0e3a-4ba4-92fb-98052fa56f1d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.113913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42914ce-0e3a-4ba4-92fb-98052fa56f1d None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.114361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 22/37] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:44.122618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42914ce-0e3a-4ba4-92fb-98052fa56f1d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:44.151187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32d515-a8f4-48b9-8f72-e501a8452095 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.196026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024ad18-7ac0-4da9-ad8d-aa27c8665d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:44.196666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ad18-7ac0-4da9-ad8d-aa27c8665d31 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.196948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ad18-7ac0-4da9-ad8d-aa27c8665d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.197226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ad18-7ac0-4da9-ad8d-aa27c8665d31 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.198287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024ad18-7ac0-4da9-ad8d-aa27c8665d31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:44.201417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024ad18-7ac0-4da9-ad8d-aa27c8665d31 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:44.240894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 16/38] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:45:43 2022] POST /identity/v3/projects => generated 301 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:44.242022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.242317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.242670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.268006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:44.269750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:44.275979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42914ce-0e3a-4ba4-92fb-98052fa56f1d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.291284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:44.339181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7bc4e-041a-410c-98fe-a46eece65f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:44.339960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7bc4e-041a-410c-98fe-a46eece65f5d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.340378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7bc4e-041a-410c-98fe-a46eece65f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.340604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7bc4e-041a-410c-98fe-a46eece65f5d None admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.341964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7bc4e-041a-410c-98fe-a46eece65f5d None admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:44.343580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7bc4e-041a-410c-98fe-a46eece65f5d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:44.346917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 23/39] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:44 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:44.347961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.348248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.348580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.372093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 17/40] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:44.374717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:44.377534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:44.381937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:44.396732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:44.488391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 24/41] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:44.514849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7d5fc-0894-4b23-9445-cf6f008c419d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.558637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:44.559289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.559618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.559904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.561033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:44.562589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:44.564458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 18/42] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:44.570241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:44.669910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:44.753559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:44.778991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ec768-8317-4c95-9780-93de23bb0072 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.788590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87242c4e-6731-4432-9b88-ccdc7b08e90b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.795972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:44.814481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:44.815123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.815406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.815683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.816770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:44.818246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:44.819796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 19/43] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:44.820624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:44.825117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:44.825834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:44.826124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:44.826409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] PATH_INFO: `/v3/domains/default/users/5da7578088e04700b6983b884a95e6ef/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:44.828371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5da7578088e04700b6983b884a95e6ef, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:44.829927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:44.845225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 25/44] 10.222.0.23 () {58 vars in 1361 bytes} [Sat Sep 10 20:45:44 2022] PUT /identity/v3/domains/default/users/5da7578088e04700b6983b884a95e6ef/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:45:44.845934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:44.960778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1f881-6275-4bb6-9dbd-70f1fe2d552b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:44.963571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a5f12-79e7-42ef-b326-1dd839785726 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:45.001452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc52a97-a883-4ba2-b01b-a7fc6db699f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:45.001969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9afc-1eff-4e72-8263-3260fb726fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:45.002151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc52a97-a883-4ba2-b01b-a7fc6db699f7 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:45.002404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc52a97-a883-4ba2-b01b-a7fc6db699f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:45.002598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9afc-1eff-4e72-8263-3260fb726fed None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:45.002687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc52a97-a883-4ba2-b01b-a7fc6db699f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:45.002883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9afc-1eff-4e72-8263-3260fb726fed None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:45.003161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9afc-1eff-4e72-8263-3260fb726fed None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:45.003364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc52a97-a883-4ba2-b01b-a7fc6db699f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:45.003824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9afc-1eff-4e72-8263-3260fb726fed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:45.004789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc52a97-a883-4ba2-b01b-a7fc6db699f7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:45.005305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9afc-1eff-4e72-8263-3260fb726fed None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:45.025596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 20/45] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Sep 10 20:45:44 2022] POST /identity/v3/roles => generated 223 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:45.041774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 26/46] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:45:44 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:47.044737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ed28-5df9-44ca-9d23-4a574f3b5251 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.045275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ed28-5df9-44ca-9d23-4a574f3b5251 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.045275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ed28-5df9-44ca-9d23-4a574f3b5251 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.046681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 21/47] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:47.052216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a62e-9e05-4b14-81a6-de791552c2d6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.052639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a62e-9e05-4b14-81a6-de791552c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.052897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a62e-9e05-4b14-81a6-de791552c2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.071060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0e60a62e-9e05-4b14-81a6-de791552c2d6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:47.072255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0e60a62e-9e05-4b14-81a6-de791552c2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:47.086013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:47.147469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 27/48] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:47.221396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e90d-b2c2-434e-8dfc-468315a0cad3 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.221719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e90d-b2c2-434e-8dfc-468315a0cad3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.222004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e90d-b2c2-434e-8dfc-468315a0cad3 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.223354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 22/49] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:47.228564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.228852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.229128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.246963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:47.248146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:47.319062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 28/50] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:47.425019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e90d-b2c2-434e-8dfc-468315a0cad3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:47.428776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:47.526289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e90d-b2c2-434e-8dfc-468315a0cad3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:47.559868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:47.560539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.560918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.561207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.572289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:47.577127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:47.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:45:47.579602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 23/51] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:47.584801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:47.590996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:47.653865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:47.663848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:47.686022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22aa28-df3c-481b-ae01-e54e61864ae7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:47.694416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441dbc-3820-4080-a40b-8693398998c9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:47.721471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:47.722113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.722395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.722672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.723333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:47.723639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:47.725028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:47.727042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:47.727677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.727965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.728246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.729374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:47.730847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:47.732406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 24/52] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:47.737558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 29/53] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:47.738755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:47.739598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:47.839876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67265b-f0ac-4860-9968-e73dc63d531a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:47.840545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ea450-b96c-419d-888f-5c19469e0d9b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:47.868608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:47.874011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:47.874235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:47.874626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.874872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:47.875011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.875749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:47.875864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.876136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:47.876677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:47.877282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:47.878181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:47.878758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:47.882572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 30/54] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:47 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:47.885928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:47.898653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:47.995161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9607-a8df-453c-8215-bcb4ec68ba53 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:48.008833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 25/55] 10.222.0.23 () {60 vars in 1140 bytes} [Sat Sep 10 20:45:47 2022] POST /identity/v3/projects => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:48.030482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a0041-5a6f-4285-a7c9-b7564281a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:48.031847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a0041-5a6f-4285-a7c9-b7564281a507 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:48.032231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a0041-5a6f-4285-a7c9-b7564281a507 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:48.032511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a0041-5a6f-4285-a7c9-b7564281a507 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:48.033752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a0041-5a6f-4285-a7c9-b7564281a507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:48.035246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a0041-5a6f-4285-a7c9-b7564281a507 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:48.036890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 31/56] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:48.043128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:48.147095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791d9a-b7b0-45e5-baef-4d14652cc4a3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:48.182452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:48.183094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:48.183357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:48.183632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:48.184363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.domain.id=default, user.id=5da7578088e04700b6983b884a95e6ef) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:48.184670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:48.186160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2df244-8c5b-4f00-9432-dd3869adaa70 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:48.195273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 26/57] 10.222.0.23 () {56 vars in 1322 bytes} [Sat Sep 10 20:45:48 2022] GET /identity/v3/role_assignments?user.id=5da7578088e04700b6983b884a95e6ef&scope.domain.id=default&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:50.156178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d035a-7cb2-4fd9-bf56-f78a964baaab None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:50.156664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d035a-7cb2-4fd9-bf56-f78a964baaab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:50.156824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d035a-7cb2-4fd9-bf56-f78a964baaab None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:50.158325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 32/58] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:50.163623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:50.164030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:50.164192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:50.182954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:50.184189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:50.226472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:50.250517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 27/59] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:50.517170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d035a-7cb2-4fd9-bf56-f78a964baaab None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:50.594828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:50.619129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d035a-7cb2-4fd9-bf56-f78a964baaab None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:50.652298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265e6a1-519b-418b-a827-f3b3d241b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:50.652922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265e6a1-519b-418b-a827-f3b3d241b43b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:50.653206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265e6a1-519b-418b-a827-f3b3d241b43b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:50.653503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265e6a1-519b-418b-a827-f3b3d241b43b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:50.654604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265e6a1-519b-418b-a827-f3b3d241b43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:50.656044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265e6a1-519b-418b-a827-f3b3d241b43b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:50.657783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 33/60] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:50.663370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:50.760553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742c2f-86d5-438b-b795-bc85c09c4866 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:50.794651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7303ff3-beae-454f-853a-cec65065b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:50.795269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303ff3-beae-454f-853a-cec65065b1c1 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:50.795550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303ff3-beae-454f-853a-cec65065b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:50.795824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303ff3-beae-454f-853a-cec65065b1c1 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:50.796502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7303ff3-beae-454f-853a-cec65065b1c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:50.798750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7303ff3-beae-454f-853a-cec65065b1c1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:50.825033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 28/61] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:45:50 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:53.961586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088156-64be-45e0-8c44-de01fe77a155 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:53.962534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088156-64be-45e0-8c44-de01fe77a155 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:53.963041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088156-64be-45e0-8c44-de01fe77a155 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:53.964871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 34/62] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:53.970102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7f6a-2ad7-464b-879b-e26bab786dc6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:53.970897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7f6a-2ad7-464b-879b-e26bab786dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:53.971391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7f6a-2ad7-464b-879b-e26bab786dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:53.998454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-585d7f6a-2ad7-464b-879b-e26bab786dc6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:53.999997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-585d7f6a-2ad7-464b-879b-e26bab786dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:54.036532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:54.087448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 29/63] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:54.283071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113cbc-5867-4c73-9520-d56969067717 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.283458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113cbc-5867-4c73-9520-d56969067717 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.283635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113cbc-5867-4c73-9520-d56969067717 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.285233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 35/64] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:54.299221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cdb96-50bd-454a-98cb-23dd762d717e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.299510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cdb96-50bd-454a-98cb-23dd762d717e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.299790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cdb96-50bd-454a-98cb-23dd762d717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.326488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7a6cdb96-50bd-454a-98cb-23dd762d717e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:54.327701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7a6cdb96-50bd-454a-98cb-23dd762d717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:54.416029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 30/65] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:54.427877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae113cbc-5867-4c73-9520-d56969067717 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:54.542212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae113cbc-5867-4c73-9520-d56969067717 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:54.566588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c734-31e4-4886-93f9-d866b22ed895 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.567194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c734-31e4-4886-93f9-d866b22ed895 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.567691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c734-31e4-4886-93f9-d866b22ed895 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.569511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 31/66] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:54.574242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.574785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.575256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.603481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:54.605740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:54.618038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:54.619089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.619570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.620042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.637323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:54.644868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:54.645810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:45:54.646580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:45:54.654692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 36/67] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:54.662180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:54.669749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:54.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 32/68] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:45:54.751065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:54.797420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:54.816447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a7e70-d648-431a-8901-e2e53d611381 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:54.855890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:54.856918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.857204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.857518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.858363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:54.858855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:54.860648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7d3d9-772a-4694-8925-9c60a9971479 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:54.871250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 37/69] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:54.877498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5cfdc-e102-4c84-8993-544d6142aeeb None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.878215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5cfdc-e102-4c84-8993-544d6142aeeb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.878755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5cfdc-e102-4c84-8993-544d6142aeeb None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.880505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 38/70] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:54.882605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5cfdc-e102-4c84-8993-544d6142aeeb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:54.895415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404999b1-8e22-4734-b3d4-36279e6ae4d4 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:54.933286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889389c-a788-4d00-9b38-1708c363d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:54.934283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889389c-a788-4d00-9b38-1708c363d433 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.934766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889389c-a788-4d00-9b38-1708c363d433 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.935318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889389c-a788-4d00-9b38-1708c363d433 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.947169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889389c-a788-4d00-9b38-1708c363d433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-6889389c-a788-4d00-9b38-1708c363d433 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:54.952488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:54.953398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:45:54.954208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:45:54.956790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 33/71] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:54.958735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:54.959258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:54.959754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:54.962505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:54.981953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:54.983168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:55.004010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5cfdc-e102-4c84-8993-544d6142aeeb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.040087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d892b-d674-4c82-8570-46ff43333968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.041152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d892b-d674-4c82-8570-46ff43333968 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.041445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d892b-d674-4c82-8570-46ff43333968 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.041918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d892b-d674-4c82-8570-46ff43333968 None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.059286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 34/72] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:55.059286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.061064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d892b-d674-4c82-8570-46ff43333968 None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.063147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d892b-d674-4c82-8570-46ff43333968 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.065077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 39/73] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:54 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:55.066062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4290-ba46-4f1b-b9ae-d9e5949426c5 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.066348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4290-ba46-4f1b-b9ae-d9e5949426c5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.066621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4290-ba46-4f1b-b9ae-d9e5949426c5 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.071937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 40/74] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:55.071937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf4290-ba46-4f1b-b9ae-d9e5949426c5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.079088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.179599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d83cda-3a68-49c4-b988-f654fac8d446 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.185984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf4290-ba46-4f1b-b9ae-d9e5949426c5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.212137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.221814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.222436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.222722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.223045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.223673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:55.223982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.224147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bda41-0a58-4e78-a757-d9ffdc85422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.224785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bda41-0a58-4e78-a757-d9ffdc85422a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.225078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bda41-0a58-4e78-a757-d9ffdc85422a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.225357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bda41-0a58-4e78-a757-d9ffdc85422a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.225473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.236289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bda41-0a58-4e78-a757-d9ffdc85422a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:55.239531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 35/75] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:55.240538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-471bda41-0a58-4e78-a757-d9ffdc85422a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:55.241150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:55.242109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:45:55.242879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 41/76] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:55.243516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.244089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.244089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.264114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:55.266155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:55.334966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 42/77] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:45:55.337022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.337202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.363423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131744a-308a-474d-8a49-3895e33cd7e3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.400937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.402143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.402621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.403468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.444206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc31a6a-1a1c-4b65-9ed3-50e3f2feb4ef None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.460420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.479314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.480918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.487418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.488093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.488379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.488657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.489832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.491289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.497525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 43/78] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:55.497525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.567798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.592079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 36/79] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:45:55.592784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.618892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b955f-91f4-4ee6-91bf-4f33bfe9dda8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.652573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.653190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.653502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.653783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.654448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:55.654764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.656783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.664595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 44/80] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:55.665383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.694791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d1d65-3b38-4584-9a38-93a0b1838f8b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.711940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.728982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.729649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.729931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.730212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.741065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:45:55.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:45:55.747010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:45:55.747812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 37/81] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:45:55.748445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.769871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a883-aa10-491c-9553-f28e3d2a774a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.783960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.810853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.811483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.811765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.812039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.854664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b7af9-c751-4563-a0d7-d75ea9c0e454 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:55.882162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.883663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.889753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:55.890410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:55.891218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:55.891505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:55.892691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:55.894187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:55.895847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 38/82] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:55.896627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:55.930559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:55.985864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 45/83] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:45:55.986714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.002306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828fea3-4e3e-456e-88a4-8163300f90c0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.005306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.037122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.037791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.038084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.038363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.039478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.040941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.042537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 39/84] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.043304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.086860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14555ea-3706-43b6-bf17-32db399d80de None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.120567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.121143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.121431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.121744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.122876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.124345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.126022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 46/85] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:55 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.126795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.142196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c813053-b502-4eba-a899-90595744d4d7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.142980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.176367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.176977 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.177257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.177559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.178216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:56.178523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.179906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.187875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 40/86] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.188810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.222941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.229117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467f157-7f62-4272-8dab-528e1523ac80 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.263921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.264786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.265082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.265364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.266230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=db0aec645bb84ac0b38455ee8fbc12af) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:56.266478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.267965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.273263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 47/87] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/role_assignments?user.id=db0aec645bb84ac0b38455ee8fbc12af&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.278476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.295124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f769a4-1e89-453d-b529-c83eb7a8d388 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.338890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.339553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.339838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.340119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.341369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.342982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.350138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 41/88] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.350138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.372137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.399901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633b469-4aed-4db5-9beb-86411b239d15 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.436890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.438204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.438548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.438893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.439615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.441142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.464147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6afc74-8582-4039-a216-40babe0e632e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.466331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.471763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 48/89] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:45:56 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:45:56.477498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.515024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.516269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.516571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.516932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.606077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe06f9-b8c9-44e0-a7f4-cb3be47b4f1a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.607149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.608698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.649773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.657931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.659195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.660233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.660723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.662938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.664947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.668042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 49/90] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.669406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.739371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 42/91] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:45:56.739553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.745326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.799758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185e26f-2b87-474a-950e-7b70eae2c879 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.849144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.849879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.850305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.850417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.851150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=5da7578088e04700b6983b884a95e6ef) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:56.851461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:56.853207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:56.861828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 50/92] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/role_assignments?user.id=5da7578088e04700b6983b884a95e6ef&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:56.862875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:56.867574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e0e6d-e592-4dc1-aa94-4009399a01b3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:56.912995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:56.913580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:56.913872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:56.914152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:56.950064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:57.009383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f585a32-0c7d-4a13-b405-3a715e48019c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:57.012603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:57.014410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:57.017177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:57.061219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:57.063067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:57.064029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:57.064497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:57.066056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:57.068742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:57.071372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 51/93] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:57.079344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:57.212176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 43/94] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:45:57.218285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:57.244584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512bc4e-5c25-4480-be9e-889dec5a364e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:57.309859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:57.315058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:57.315700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:57.315984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:57.316262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:57.316997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] RBAC: Adding query filter params (role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=db0aec645bb84ac0b38455ee8fbc12af) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:57.317307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:57.318822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:57.327889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 52/95] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:45:57 2022] GET /identity/v3/role_assignments?user.id=db0aec645bb84ac0b38455ee8fbc12af&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:57.397174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075abcf-e74b-4189-8076-caf86e00576a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:57.409904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:57.444424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc74ef8-6529-47fb-8210-bd07f36c7d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:57.445085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc74ef8-6529-47fb-8210-bd07f36c7d14 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:57.445376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc74ef8-6529-47fb-8210-bd07f36c7d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:57.445703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc74ef8-6529-47fb-8210-bd07f36c7d14 None admin] PATH_INFO: `/v3/projects/ce81cf7644144e3595ded4682c5b0ebf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:57.446897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc74ef8-6529-47fb-8210-bd07f36c7d14 None admin] RBAC: Authorizing `identity:get_project(project_id=ce81cf7644144e3595ded4682c5b0ebf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:57.448395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc74ef8-6529-47fb-8210-bd07f36c7d14 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:57.455714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 44/96] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:45:57 2022] GET /identity/v3/projects/ce81cf7644144e3595ded4682c5b0ebf => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:45:57.457192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:57.571508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea3042-d2a0-4b5b-b127-4907d79cce65 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:57.607135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:57.607758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:57.608041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:57.608321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:57.609063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=ce81cf7644144e3595ded4682c5b0ebf, user.id=db0aec645bb84ac0b38455ee8fbc12af) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:45:57.609991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:57.611677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751519c-096f-439a-8986-8f00521d57fa None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:57.617508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 53/97] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:45:57 2022] GET /identity/v3/role_assignments?user.id=db0aec645bb84ac0b38455ee8fbc12af&scope.project.id=ce81cf7644144e3595ded4682c5b0ebf&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:59.768616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265d089-1b60-42a3-9786-0f1f57246090 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:59.769130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265d089-1b60-42a3-9786-0f1f57246090 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:59.769205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265d089-1b60-42a3-9786-0f1f57246090 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:59.770582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 45/98] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:59.776325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488b8d4-01fa-4bd2-9e5f-2866b279fb3e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:59.776763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488b8d4-01fa-4bd2-9e5f-2866b279fb3e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:59.776936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488b8d4-01fa-4bd2-9e5f-2866b279fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:59.797794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f488b8d4-01fa-4bd2-9e5f-2866b279fb3e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:59.799209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f488b8d4-01fa-4bd2-9e5f-2866b279fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:59.871206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeffaf1-1f8a-44a3-a7e4-5d2cbf2fa77f None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:59.871535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeffaf1-1f8a-44a3-a7e4-5d2cbf2fa77f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:59.871774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeffaf1-1f8a-44a3-a7e4-5d2cbf2fa77f None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:59.873122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 46/99] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:45:59.878144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e4fe-34db-4df4-b49b-926c3d727505 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:59.878435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e4fe-34db-4df4-b49b-926c3d727505 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:59.878717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e4fe-34db-4df4-b49b-926c3d727505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:59.885836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:59.910863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 54/100] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:45:59.919273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bbc3e4fe-34db-4df4-b49b-926c3d727505 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:59.920503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bbc3e4fe-34db-4df4-b49b-926c3d727505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:00.035280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aecb7-534e-491d-81b5-c6ea419bbb5a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.035598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aecb7-534e-491d-81b5-c6ea419bbb5a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.035851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aecb7-534e-491d-81b5-c6ea419bbb5a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.037210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 55/101] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:46:00.042715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c85b7-0583-48b2-8478-3ed7252de712 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.042991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c85b7-0583-48b2-8478-3ed7252de712 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.043274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c85b7-0583-48b2-8478-3ed7252de712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.051647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 47/102] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:00.079876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-824c85b7-0583-48b2-8478-3ed7252de712 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:00.081116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-824c85b7-0583-48b2-8478-3ed7252de712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:00.179762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 56/103] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:00.237893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0683f-1f19-48c7-8055-54588083a77d None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.238187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0683f-1f19-48c7-8055-54588083a77d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.238468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0683f-1f19-48c7-8055-54588083a77d None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.239831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 48/104] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:00.245110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87d98-ac46-4c70-8bab-e4af5e9758d9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.245403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87d98-ac46-4c70-8bab-e4af5e9758d9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.245721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87d98-ac46-4c70-8bab-e4af5e9758d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.265176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-16a87d98-ac46-4c70-8bab-e4af5e9758d9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:00.267118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-16a87d98-ac46-4c70-8bab-e4af5e9758d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:00.335494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c00a-ba42-4b2f-9f12-28bff2ec6508 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.335794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c00a-ba42-4b2f-9f12-28bff2ec6508 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.335999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c00a-ba42-4b2f-9f12-28bff2ec6508 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.336373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 57/105] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:00.338026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 49/106] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:00.343431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.343780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.344665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.363146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:00.364339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:00.416727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c00a-ba42-4b2f-9f12-28bff2ec6508 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:00.433807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:00.475087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 58/107] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:00.486517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:00.613965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:00.616160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c00a-ba42-4b2f-9f12-28bff2ec6508 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:00.686046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:00.686698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.686982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.687262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.705692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:00.710596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:00.711626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 50/108] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:00.712665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:00.715162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c966a61-bf49-4dd1-accb-6eaad8ef2a55 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:00.757802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:00.758552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.758909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.759260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.773974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:00.779993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:00.782182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 59/109] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:00.784061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:00.838217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a1db-3a0d-41db-8654-2965676f8d14 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:00.840650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:00.877213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:00.877811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.878100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.878383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.896741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:00.902152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:00.902970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:00.903739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 51/110] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:00.904355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:00.910815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01263-bd48-4780-9385-e23d3856fda6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:00.927929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:00.950820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:00.951745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:00.952057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:00.952392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:00.963084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:00.967847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:00.969227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:00.970163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 60/111] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:00.971667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.015848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8375c-fabe-42b3-a611-d48ec43c057e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.053188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.053867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.054172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.054438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.055175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:01.055489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.056950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.065836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 52/112] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.066509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.076898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.082043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b812ae-539a-4de5-b9e3-4248e0be155c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.133784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.134629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.135060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.135476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.136135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:01.136453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.138065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.148854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 61/113] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.149712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.159674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.180717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7f3be-07e2-4578-8c5c-8734e1c5b6af None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.216166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.216805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.217086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.217358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.218489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.220415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.222011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 53/114] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.222771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.251707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee55ceb-33c9-4a99-916a-cf4d59d11785 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.286568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.287185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.287473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.287752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.288446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:01.288756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.290171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.299207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 62/115] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.299935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.324456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c63e8-9dda-47fd-992e-459bc936e0c0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.325520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.360342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.360624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.360897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.361577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:01.361882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.363251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.371047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 54/116] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.375779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.391700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.403632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea6212-c5fb-480e-a516-f6c3516ffaec None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.439508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0baa1-8664-4db0-a433-c27d8d7d9053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.440154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0baa1-8664-4db0-a433-c27d8d7d9053 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.440442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0baa1-8664-4db0-a433-c27d8d7d9053 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.440722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0baa1-8664-4db0-a433-c27d8d7d9053 None admin] PATH_INFO: `/v3/users/ca23ea22e99a4c88acff38ef4ed51102` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.455046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0baa1-8664-4db0-a433-c27d8d7d9053 None admin] RBAC: Authorizing `identity:get_user(user_id=ca23ea22e99a4c88acff38ef4ed51102)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.456571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0baa1-8664-4db0-a433-c27d8d7d9053 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.458342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 63/117] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/users/ca23ea22e99a4c88acff38ef4ed51102 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.459457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e048d0-b23c-45fd-a95d-3f48b520664e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.459746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e048d0-b23c-45fd-a95d-3f48b520664e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.460027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e048d0-b23c-45fd-a95d-3f48b520664e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.461355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 64/118] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:01.461999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e048d0-b23c-45fd-a95d-3f48b520664e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.476262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48d713-e2f2-4838-9f54-4bee0e339a1b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.510026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.510663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.510953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.511237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.512592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.514066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.516466 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 55/119] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.517398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.546386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.566024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e048d0-b23c-45fd-a95d-3f48b520664e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.599066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.599945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.600559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.600894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.601179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.602051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.603502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.618049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb22ed-7b5a-4b44-88ac-045a02fb175f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.631272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 65/120] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:46:01 2022] POST /identity/v3/projects => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:01.634335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.657911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.658497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.658781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.659059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.660584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.662109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.663969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 56/121] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.669358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.748731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178f92-25f3-4cb2-9a9d-ca4b49c8a5ce None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.788254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d210991-3469-4c90-9985-e8a09760a8bb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:01.788801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d603f4-0cae-41c2-afd6-c2f821d4b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.789498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d603f4-0cae-41c2-afd6-c2f821d4b4dd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.789786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d603f4-0cae-41c2-afd6-c2f821d4b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.790068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d603f4-0cae-41c2-afd6-c2f821d4b4dd None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.791256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d603f4-0cae-41c2-afd6-c2f821d4b4dd None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.792758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d603f4-0cae-41c2-afd6-c2f821d4b4dd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:01.799277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 66/122] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:01.799277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.799277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.799277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.803317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.821185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:01.822509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:01.834105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:01.834740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:01.835022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:01.835296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:01.856862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:01.921515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 67/123] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:01.921515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:01.925365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:01.926968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.046084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 57/124] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:02.046598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77041c1-d51d-40ad-950a-acc03f335a71 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.047709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.082213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:02.088796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.089811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.090204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.090584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.091838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.093425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.096460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 68/125] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:01 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:02.097197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.112606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:02.169417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff70c-5a67-4187-acda-69c89c70e11f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.211232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e65525-da90-45b4-bf02-d989a7fec58e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.215185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.215841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.216130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.216409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.217587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.220472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.222668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 58/126] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:02 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:02.223786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.247485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.248126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.248407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.248683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.249877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.252148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.254816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 69/127] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:02 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:02.254926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.328301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:02.334356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b485-db1a-43e2-a0b5-bd9e0da70d4a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.351860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:02.368763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e85928-a3d1-4c2a-8e4b-6f683298f66e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.370375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.371907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.372277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.372635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.373942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.376000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.378406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 59/128] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:02 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:02.379028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.410433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.411098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.411380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.411652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17/users/db0aec645bb84ac0b38455ee8fbc12af/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.413697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] RBAC: Authorizing `identity:create_grant(project_id=c63e8d69313f433695ce236bb79eba17, user_id=db0aec645bb84ac0b38455ee8fbc12af, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.415105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.428691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 70/129] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:02 2022] PUT /identity/v3/projects/c63e8d69313f433695ce236bb79eba17/users/db0aec645bb84ac0b38455ee8fbc12af/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:02.429296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.495768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565432c-503a-4870-986f-64fb099e1470 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.547237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.548650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.549690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.550084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.568501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:02.573758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:02.574703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:02.575608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 60/130] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:02.576709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.580161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da2df6-cab2-4923-898c-11fa0b692a6a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.621306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:02.630235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:02.631573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.632178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.632881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.633702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17/users/5da7578088e04700b6983b884a95e6ef/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.636518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c63e8d69313f433695ce236bb79eba17, user_id=5da7578088e04700b6983b884a95e6ef, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.639202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.655869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 71/131] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:02 2022] PUT /identity/v3/projects/c63e8d69313f433695ce236bb79eba17/users/5da7578088e04700b6983b884a95e6ef/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:02.658358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.705508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b511b-a4d2-45f9-9366-41ed30c12ff9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.782970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.783715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.784008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.784291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17/users/db0aec645bb84ac0b38455ee8fbc12af/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.790544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] RBAC: Authorizing `identity:create_grant(project_id=c63e8d69313f433695ce236bb79eba17, user_id=db0aec645bb84ac0b38455ee8fbc12af, role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.790750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17eff-5c40-41a5-978f-7748bf2c85d9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:02.792191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.834539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:02.835168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:02.835449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:02.835720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:02.836457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 61/132] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:02 2022] PUT /identity/v3/projects/c63e8d69313f433695ce236bb79eba17/users/db0aec645bb84ac0b38455ee8fbc12af/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:02.836571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] RBAC: Adding query filter params (role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=ca23ea22e99a4c88acff38ef4ed51102) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:02.836889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:02.837415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:02.838392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:02.845662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 72/133] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:02 2022] GET /identity/v3/role_assignments?user.id=ca23ea22e99a4c88acff38ef4ed51102&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:03.029637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831f9d-b074-4abd-beef-870216ba9843 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:03.034858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:03.079844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:03.080538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:03.080834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:03.081119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:03.081830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:03.082142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:03.083554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:03.098435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 62/134] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:03.104357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:03.163520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:03.226044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f89d9-f7ce-4f75-a6a6-1df5996d3d67 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:03.268329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:03.269333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:03.269852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:03.270333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:03.271669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:03.273340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:03.275189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 73/135] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:03 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:03.280721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:03.391130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622be87-20d0-4c8d-90ec-d057fd47b759 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:03.434281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79530f11-32bc-4661-a35e-5e886476a05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:03.435306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79530f11-32bc-4661-a35e-5e886476a05c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:03.435795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79530f11-32bc-4661-a35e-5e886476a05c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:03.436285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79530f11-32bc-4661-a35e-5e886476a05c None admin] PATH_INFO: `/v3/projects/ce81cf7644144e3595ded4682c5b0ebf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:03.437716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79530f11-32bc-4661-a35e-5e886476a05c None admin] RBAC: Authorizing `identity:get_project(project_id=ce81cf7644144e3595ded4682c5b0ebf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:03.439780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79530f11-32bc-4661-a35e-5e886476a05c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:03.441766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 63/136] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:03 2022] GET /identity/v3/projects/ce81cf7644144e3595ded4682c5b0ebf => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:03.447606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:03.575443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaec05-ffdf-4fba-b29a-68268865a9cf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:03.599072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:03.620118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f50f9d-8cd2-42bb-b7e3-ffb4bf797d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:03.620785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f50f9d-8cd2-42bb-b7e3-ffb4bf797d22 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:03.621060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f50f9d-8cd2-42bb-b7e3-ffb4bf797d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:03.621333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f50f9d-8cd2-42bb-b7e3-ffb4bf797d22 None admin] PATH_INFO: `/v3/projects/ce81cf7644144e3595ded4682c5b0ebf/users/db0aec645bb84ac0b38455ee8fbc12af/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:03.623280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f50f9d-8cd2-42bb-b7e3-ffb4bf797d22 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce81cf7644144e3595ded4682c5b0ebf, user_id=db0aec645bb84ac0b38455ee8fbc12af, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:03.624740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f50f9d-8cd2-42bb-b7e3-ffb4bf797d22 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:03.652334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 74/137] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:03 2022] PUT /identity/v3/projects/ce81cf7644144e3595ded4682c5b0ebf/users/db0aec645bb84ac0b38455ee8fbc12af/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:04.586298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302d38d-a92a-4ccb-b7fc-278d909c817b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:04.587325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302d38d-a92a-4ccb-b7fc-278d909c817b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:04.587833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302d38d-a92a-4ccb-b7fc-278d909c817b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:04.589390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 64/138] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:04.595375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee7537-3896-42b4-a4fe-dc176c4cee49 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:04.595797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee7537-3896-42b4-a4fe-dc176c4cee49 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:04.596016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee7537-3896-42b4-a4fe-dc176c4cee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:04.622794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5fee7537-3896-42b4-a4fe-dc176c4cee49 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:04.623996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5fee7537-3896-42b4-a4fe-dc176c4cee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:04.712557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 75/139] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:05.236008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302d38d-a92a-4ccb-b7fc-278d909c817b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:05.249490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc67f-6358-498a-9661-755b87e5f3e6 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:05.249490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc67f-6358-498a-9661-755b87e5f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:05.249490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc67f-6358-498a-9661-755b87e5f3e6 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:05.249490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 76/140] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:05.255197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:05.255480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:05.255779 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:05.277221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:05.278686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:05.290862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:05.414305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302d38d-a92a-4ccb-b7fc-278d909c817b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:05.415245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 77/141] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:05.472164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a126c-e41f-4ac1-a4b5-f83a94492613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:05.472822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a126c-e41f-4ac1-a4b5-f83a94492613 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:05.473116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a126c-e41f-4ac1-a4b5-f83a94492613 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:05.473394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a126c-e41f-4ac1-a4b5-f83a94492613 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:05.475040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a126c-e41f-4ac1-a4b5-f83a94492613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:05.476525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a126c-e41f-4ac1-a4b5-f83a94492613 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:05.478556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 65/142] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:05.483820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:05.559462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:05.646416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010bb93-3b12-4476-86a9-229c514d1d79 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:05.698121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc8a1c-b144-40c4-8ecc-b98651025fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:05.698732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8a1c-b144-40c4-8ecc-b98651025fcb None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:05.699010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8a1c-b144-40c4-8ecc-b98651025fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:05.699279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8a1c-b144-40c4-8ecc-b98651025fcb None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:05.699925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8a1c-b144-40c4-8ecc-b98651025fcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:05.701361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8a1c-b144-40c4-8ecc-b98651025fcb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:05.744008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 78/143] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:05 2022] POST /identity/v3/users => generated 283 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:05.981013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e8f74-c74f-47a3-bc54-36e4729c44d3 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:05.982011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e8f74-c74f-47a3-bc54-36e4729c44d3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:05.982505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e8f74-c74f-47a3-bc54-36e4729c44d3 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:05.984132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 66/144] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:05.991039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2cff1-924d-4a1e-8dd6-e0e1e2278945 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:05.991570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2cff1-924d-4a1e-8dd6-e0e1e2278945 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:05.992071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2cff1-924d-4a1e-8dd6-e0e1e2278945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.014172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-06f2cff1-924d-4a1e-8dd6-e0e1e2278945 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:06.020315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-06f2cff1-924d-4a1e-8dd6-e0e1e2278945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:06.025746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e8f74-c74f-47a3-bc54-36e4729c44d3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.107051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 79/145] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:06.111343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d8543-6609-4cd7-b149-dd407395a791 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.111869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d8543-6609-4cd7-b149-dd407395a791 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.112247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d8543-6609-4cd7-b149-dd407395a791 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.113832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 80/146] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:06.119055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.119268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.119928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.158761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:06.160015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:06.171386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e8f74-c74f-47a3-bc54-36e4729c44d3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:06.175223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:06.197244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:06.246879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:06.247898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.248363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.248817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.270893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:06.278833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:06.279800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:06.280609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:06.282978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 67/147] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:06.289367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.305012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 81/148] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:06.414141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7203-43b2-4f6a-bbef-5915d0a6694b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:06.451621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:06.452500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.453579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.453870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.454536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:06.454775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.454884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:06.456289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:06.472756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 68/149] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:06.479706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.531933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:06.634916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb19b6-de55-4590-84bb-203a53ebccaf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:06.647304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc185485-1792-42c9-93cc-a6b676234471 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:06.678274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:06.678937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.679217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.679587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.690836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3f62b-7563-4aed-b2b1-d90198fa18ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:06.691487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3f62b-7563-4aed-b2b1-d90198fa18ce None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.691658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:06.691809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3f62b-7563-4aed-b2b1-d90198fa18ce None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.692049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3f62b-7563-4aed-b2b1-d90198fa18ce None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.693224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3f62b-7563-4aed-b2b1-d90198fa18ce None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:06.694729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3f62b-7563-4aed-b2b1-d90198fa18ce None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:06.696520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 69/150] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:06.696619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:06.697418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:06.698594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:06.699251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79b84d-9ddf-465a-a50e-f9eefb7b4f31 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.699251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79b84d-9ddf-465a-a50e-f9eefb7b4f31 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.699755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 82/151] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:06.699913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79b84d-9ddf-465a-a50e-f9eefb7b4f31 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.700059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.700196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 70/152] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:06.705616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.705909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.706196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.720472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:06.726273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:06.727494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:06.816937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db12e0-1fed-4fec-8985-79f156365fb0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:06.822124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 71/153] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:06.823038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.854667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:06.855425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:06.855709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:06.855980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:06.869523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:06.869715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:06.874136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:06.875129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 83/154] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:06.876595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:06.966316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c97d0a-3e44-4108-9d55-dfdedb63953d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.010226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.011683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.012160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.012692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.013610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:07.014269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.015079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:07.020681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.043874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 72/155] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:07.045150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.071331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e81c0-a4e7-405b-9d01-cfa7bacf6e6b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.143610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.143610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.143610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.143969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.193646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:07.214900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffd120-d597-4f69-ae0c-d26b624d390c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.237967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.240324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.262740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.263517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.263912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.264354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.265383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:07.266337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.267870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.279728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 73/156] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:07.280809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.345856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:07.384365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 84/157] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:07.385247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.409276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259cb31-0f30-4a00-9343-1d9bb68e5b5b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.458204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.458869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.459160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.459442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] PATH_INFO: `/v3/users/5da7578088e04700b6983b884a95e6ef` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.460644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] RBAC: Authorizing `identity:get_user(user_id=5da7578088e04700b6983b884a95e6ef)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.462152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.468319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 74/158] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/users/5da7578088e04700b6983b884a95e6ef => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:07.468319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.497267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:07.516233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24449cc-9976-4e93-afcf-7bbc77944a68 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.580343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc15b-2f7b-4056-a282-a939f9bbc43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.580991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc15b-2f7b-4056-a282-a939f9bbc43e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.581274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc15b-2f7b-4056-a282-a939f9bbc43e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.581578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc15b-2f7b-4056-a282-a939f9bbc43e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.608040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdc15b-2f7b-4056-a282-a939f9bbc43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e2bdc15b-2f7b-4056-a282-a939f9bbc43e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:07.619085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:07.620005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:07.620854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 85/159] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:07.621507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628204cb-e2b1-476a-ac2d-21a628849378 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.622385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628204cb-e2b1-476a-ac2d-21a628849378 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.622385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628204cb-e2b1-476a-ac2d-21a628849378 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.623861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 86/160] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:07.624648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628204cb-e2b1-476a-ac2d-21a628849378 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.665211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4ee5-8e7e-4c7f-9bb8-6b003a3b0bfa None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.673246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:07.735115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.735941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.736221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.736497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] PATH_INFO: `/v3/users/ca23ea22e99a4c88acff38ef4ed51102` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.739155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] RBAC: Authorizing `identity:get_user(user_id=ca23ea22e99a4c88acff38ef4ed51102)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.741562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.746789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 75/161] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/users/ca23ea22e99a4c88acff38ef4ed51102 => generated 289 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:07.748011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.806291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628204cb-e2b1-476a-ac2d-21a628849378 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.855718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:07.869245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.870143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.870906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.871321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.872580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.874782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.887268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 87/162] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:07.887268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.943433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c832ca3-ae77-4adc-b025-a415b49e17df None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:08.009838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:08.010373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:08.010893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:08.010893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:08.044870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adddd9-0e95-4cd0-8d09-7be968d2f629 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:08.048913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:08.115307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:08.115930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:08.116210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:08.116475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:08.117137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:08.117437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:08.118839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdfec7-d5f0-4d6e-9664-5d6f571d5c1d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:08.135307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 88/163] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:08.136559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:08.136836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:08.137113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:08.152554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:08.154163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:08.175491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:08.176684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:08.285929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 89/164] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:08.287119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:08.309811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:08.333937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 76/165] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:08.334838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:08.502146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:08.504830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8b32e-9e90-47d8-9c5c-858385665bdf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:08.558836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:08.559796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:08.560318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:08.560966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:08.566413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:08.568315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:08.570245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 90/166] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:08 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:08.571600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:08.584774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04034c9e-d085-4b71-85e1-c74a72a43ad3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:08.651494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:08.652235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:08.652524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:08.652807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:08.653571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=db0aec645bb84ac0b38455ee8fbc12af) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:08.653886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:08.655354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:08.667072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 77/167] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:08 2022] GET /identity/v3/role_assignments?user.id=db0aec645bb84ac0b38455ee8fbc12af&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 608 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:08.668064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:08.685496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634235-9666-462f-a44d-6ec1cb5fdb3d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:08.737347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:08.738061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:08.738344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:08.738618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:08.740507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:08.742174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:08.750289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 91/168] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:08 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:08.751504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:08.791662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:08.809913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:08.937718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535d1eb-492b-4920-ade6-d0027396e8c8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:09.016091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc4c0a-4ee3-40cc-9d73-d324aa0d537d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:09.019777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:09.020487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.020772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.021047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.023177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:09.024630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:09.031773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 78/169] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:08 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:09.032638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:09.087961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:09.088559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.088874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.089166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17/users/ca23ea22e99a4c88acff38ef4ed51102/roles/d20f3e89a1aa4fa58b9d6cedc13ed5f0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.091743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] RBAC: Authorizing `identity:create_grant(project_id=c63e8d69313f433695ce236bb79eba17, user_id=ca23ea22e99a4c88acff38ef4ed51102, role_id=d20f3e89a1aa4fa58b9d6cedc13ed5f0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:09.094319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:09.117526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 92/170] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:08 2022] PUT /identity/v3/projects/c63e8d69313f433695ce236bb79eba17/users/ca23ea22e99a4c88acff38ef4ed51102/roles/d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:09.121312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:09.224967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785cbad-92ab-472f-9e71-c905d1bbaa53 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:09.256271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:09.275563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:09.296135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b864e-3973-4909-889e-5a1b3313f004 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:09.313645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:09.314311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.314598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.314875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.337660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:09.347004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:09.347939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:09.348746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:09.355338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 79/171] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:09.356377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:09.383711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ccc43-9671-4a52-b2c4-267ee2f7d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:09.384404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ccc43-9671-4a52-b2c4-267ee2f7d116 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.384692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ccc43-9671-4a52-b2c4-267ee2f7d116 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.384971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ccc43-9671-4a52-b2c4-267ee2f7d116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.517992 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ccc43-9671-4a52-b2c4-267ee2f7d116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:09.519562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ccc43-9671-4a52-b2c4-267ee2f7d116 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:09.559913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce17dc-147a-472e-82be-c3605d40a8bb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:09.612301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:09.614117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.614409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.614687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.615428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=5da7578088e04700b6983b884a95e6ef) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:09.615738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:09.617193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:09.632859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 80/172] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:09 2022] GET /identity/v3/role_assignments?user.id=5da7578088e04700b6983b884a95e6ef&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:09.633588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:09.697154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:09.711107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:09.720659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 93/173] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:09.722012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cc73f-9a60-46d0-9036-ff5bd203b5e8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.722302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cc73f-9a60-46d0-9036-ff5bd203b5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.722577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cc73f-9a60-46d0-9036-ff5bd203b5e8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.724123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 94/174] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:09.733800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cc73f-9a60-46d0-9036-ff5bd203b5e8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:09.824223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d94b8-e8b9-4698-b8e5-4363bc8c1fc2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:09.869052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:09.870078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.870575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.871060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.871940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:09.872452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:09.877317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29340797-95f9-4a81-bf7a-4a1b41804ff6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:09.901628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 81/175] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:09.901628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:09.901628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:09.901628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:09.938194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:09.939483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:09.968931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cc73f-9a60-46d0-9036-ff5bd203b5e8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:10.017231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fc2db-4791-45bf-9ba8-3d3b42aa291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:10.017995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fc2db-4791-45bf-9ba8-3d3b42aa291a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.018274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fc2db-4791-45bf-9ba8-3d3b42aa291a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.018544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fc2db-4791-45bf-9ba8-3d3b42aa291a None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.019754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fc2db-4791-45bf-9ba8-3d3b42aa291a None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:10.021592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fc2db-4791-45bf-9ba8-3d3b42aa291a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:10.023661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 95/176] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:09 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:10.024912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4111aeb-f704-49e6-a671-2827bafcfa27 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.025191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4111aeb-f704-49e6-a671-2827bafcfa27 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.025477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4111aeb-f704-49e6-a671-2827bafcfa27 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.027479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 96/177] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:10.028213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4111aeb-f704-49e6-a671-2827bafcfa27 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:10.038377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 82/178] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:10.039257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:10.074299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:10.139596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:10.167054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9dc9b-d1de-45fe-99ae-1e21be6c0ffc None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:10.208506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:10.209136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.209420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.209723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.210521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] RBAC: Adding query filter params (role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=db0aec645bb84ac0b38455ee8fbc12af) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:10.210839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:10.212278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d98f3-bc7a-47a2-882d-b820fc8bdfb4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:10.215129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4111aeb-f704-49e6-a671-2827bafcfa27 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:10.218778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 83/179] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/role_assignments?user.id=db0aec645bb84ac0b38455ee8fbc12af&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:10.225499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.225499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.225499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.257731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:10.259981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:10.297663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b97059-edf2-4abf-816d-4ffd6a39158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:10.298332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97059-edf2-4abf-816d-4ffd6a39158c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.298614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97059-edf2-4abf-816d-4ffd6a39158c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.298888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97059-edf2-4abf-816d-4ffd6a39158c None admin] PATH_INFO: `/v3/users/db0aec645bb84ac0b38455ee8fbc12af` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.303154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b97059-edf2-4abf-816d-4ffd6a39158c None admin] RBAC: Authorizing `identity:get_user(user_id=db0aec645bb84ac0b38455ee8fbc12af)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:10.304641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b97059-edf2-4abf-816d-4ffd6a39158c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:10.306371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 97/180] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/users/db0aec645bb84ac0b38455ee8fbc12af => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:10.319451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a941-19a2-480e-b55e-4338ce090863 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.319813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a941-19a2-480e-b55e-4338ce090863 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.320031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a941-19a2-480e-b55e-4338ce090863 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.321421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 98/181] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:10 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:10.327552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545fde5-589a-4055-a88d-633fa788719f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.327851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545fde5-589a-4055-a88d-633fa788719f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.328647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545fde5-589a-4055-a88d-633fa788719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.370804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6545fde5-589a-4055-a88d-633fa788719f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:10.372081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6545fde5-589a-4055-a88d-633fa788719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:10.424416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 84/182] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:10.425668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:10.455328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:10.495571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 99/183] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:10.496426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545fde5-589a-4055-a88d-633fa788719f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:10.537235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c336d2-4a95-47cd-9a7b-5606d6d8a250 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:10.594457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:10.604503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:10.606559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.606849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.607130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.665484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545fde5-589a-4055-a88d-633fa788719f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:10.704030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:10.704606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.704894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.705603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.716381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:10.722474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:10.723498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:10.724476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 100/184] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:10.731277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:10.751048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:10.752516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:10.792983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:10.874319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1782a-adf2-4890-8dae-75360f79df42 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:10.899363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:10.910366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 85/185] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:10.910366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:10.924321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:10.924996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:10.925277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:10.925575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:10.926247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:10.926551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:10.927945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:10.936867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 101/186] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:10.936867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.019207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed36bf-f749-4ff4-ac64-a441d6e31150 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.044934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.058545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.059179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.059459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.059733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.059839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5f0d-e8d4-4703-8ff6-f8611939ac36 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.070511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:11.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:11.076544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:11.077336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 86/187] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:11.078609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.104042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.104766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.105054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.105332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.106480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.107940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.109948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 102/188] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:11.110785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.163566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.197736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06ba03-d281-46c2-b453-c17011a62054 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.217987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db8329-856c-43ae-9df4-894715ed1a6d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.236178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.236823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.237109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.237381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] PATH_INFO: `/v3/projects/ce81cf7644144e3595ded4682c5b0ebf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.238528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] RBAC: Authorizing `identity:get_project(project_id=ce81cf7644144e3595ded4682c5b0ebf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.239981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.241629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 87/189] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/projects/ce81cf7644144e3595ded4682c5b0ebf => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:11.242392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.261369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.262240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.262671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.263087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] PATH_INFO: `/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.278040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] RBAC: Authorizing `identity:get_user(user_id=d64cd8d1f3ed4965859cd2f53b6497a0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.279730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.282086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 103/190] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:11.283043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.304292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.352687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1241534-801e-4555-afc8-2f6e27cff93d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.389686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.390940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.391240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.391750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.392298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:11.392601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.394002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.401566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3284-e051-4a0a-baae-e4553a25cd39 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.402219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 88/191] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:11.402856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.415402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.443332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.443941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.444219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.444491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.445138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.446646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.473525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 104/192] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:11 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:11.475596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.526275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169acf-cabd-4338-8d12-f805284dcc70 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.543745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.573155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.573980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.574815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.574815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.575610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=ce81cf7644144e3595ded4682c5b0ebf, user.id=db0aec645bb84ac0b38455ee8fbc12af) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:11.576195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.577892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.586216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 89/193] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/role_assignments?user.id=db0aec645bb84ac0b38455ee8fbc12af&scope.project.id=ce81cf7644144e3595ded4682c5b0ebf&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:11.595752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.636210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf006cae-2154-4fd6-bf95-37eaf80b5a6a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.708266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39be412-6bba-4513-96aa-87e08d706646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.708890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39be412-6bba-4513-96aa-87e08d706646 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.709173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39be412-6bba-4513-96aa-87e08d706646 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.709447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39be412-6bba-4513-96aa-87e08d706646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.749099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.772296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dda8a3-ff32-4e5c-ab08-13a20e9e31fd None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:11.801615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39be412-6bba-4513-96aa-87e08d706646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.803358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39be412-6bba-4513-96aa-87e08d706646 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.824487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e7095-a6f8-4876-807b-e85befab7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:11.825615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e7095-a6f8-4876-807b-e85befab7015 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.826104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e7095-a6f8-4876-807b-e85befab7015 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.826580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e7095-a6f8-4876-807b-e85befab7015 None admin] PATH_INFO: `/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.828021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e7095-a6f8-4876-807b-e85befab7015 None admin] RBAC: Authorizing `identity:get_user(user_id=d64cd8d1f3ed4965859cd2f53b6497a0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:11.829967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e7095-a6f8-4876-807b-e85befab7015 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:11.832571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 90/194] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:11.833996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a201-995d-4c89-a6bd-8105ecf5959e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.834490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a201-995d-4c89-a6bd-8105ecf5959e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.834964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a201-995d-4c89-a6bd-8105ecf5959e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.836391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 91/195] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:11.841369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a201-995d-4c89-a6bd-8105ecf5959e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:11.902840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:11.948936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 105/196] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:11.950474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:11.950652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:11.950904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:11.969987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:11.971207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:11.987271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a201-995d-4c89-a6bd-8105ecf5959e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:12.023988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:12.024639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:12.024917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:12.025188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:12.041500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:12.058637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 106/197] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:12.059714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:12.137059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:12.138577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:12.206579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:12.238667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5017db-a9de-4b7e-938e-a2ad75c9c1f1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:12.288568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:12.289611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:12.290105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:12.290596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:12.291910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:12.293581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:12.295626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 107/198] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:12 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:12.301350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:12.328992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 92/199] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:12.334364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:12.394752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:12.459891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc3ba8-4d87-4c98-8691-05f1345cc447 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:12.491509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83202b-6110-4bba-9084-9f5e2820e541 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:12.508031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:12.509007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:12.509821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:12.510178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:12.511272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=d64cd8d1f3ed4965859cd2f53b6497a0) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:12.511650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:12.513825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:12.523168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 108/200] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:12 2022] GET /identity/v3/role_assignments?user.id=d64cd8d1f3ed4965859cd2f53b6497a0&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:12.524386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:12.534570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:12.535234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:12.535516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:12.535796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:12.537855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:12.539374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:12.541905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 93/201] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:12 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:12.548974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:12.574899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:12.689047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba47d3-cd43-4172-9e94-f1024fb71375 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:12.761450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:12.762115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:12.762398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:12.762669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:12.763399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] RBAC: Adding query filter params (role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=d64cd8d1f3ed4965859cd2f53b6497a0) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:12.763712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:12.765433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:12.773716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 94/202] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:12 2022] GET /identity/v3/role_assignments?user.id=d64cd8d1f3ed4965859cd2f53b6497a0&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:12.775235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8547-a80c-4214-8c9f-4019eba3e2b0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:12.817860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:12.833437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:12.834277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:12.834633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:12.835130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:12.849836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:12.855259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:12.856103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:12.857367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 109/203] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:12.863490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:12.997335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:13.029485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c4205-f94b-4ab1-8abc-c446917a6468 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:13.072808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:13.073488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:13.073786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:13.074110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:13.074790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:13.075099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:13.076621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:13.086249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 95/204] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:13.091789 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:13.208751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8c644-8466-43b1-b444-0dc026609025 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:13.256668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:13.257274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:13.257536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:13.257806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] PATH_INFO: `/v3/users/ca23ea22e99a4c88acff38ef4ed51102` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:13.258954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] RBAC: Authorizing `identity:get_user(user_id=ca23ea22e99a4c88acff38ef4ed51102)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:13.260427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:13.262253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 110/205] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:13 2022] GET /identity/v3/users/ca23ea22e99a4c88acff38ef4ed51102 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:13.268456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:13.427664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd52a1-01b3-4349-ba05-6a5070fb09cc None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:13.474668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:13.475293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:13.475581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:13.475861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:13.490682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:13.573691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:13.575175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:13.725620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 96/206] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:13.731214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:13.813655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:13.879745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c6a1-54f1-4641-81f1-07907a6c811e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:13.930508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dc4ef-3111-4f57-813d-929779a8e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:13.931159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dc4ef-3111-4f57-813d-929779a8e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:13.931487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dc4ef-3111-4f57-813d-929779a8e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:13.931766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dc4ef-3111-4f57-813d-929779a8e6c6 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:13.932932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503dc4ef-3111-4f57-813d-929779a8e6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:13.934499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503dc4ef-3111-4f57-813d-929779a8e6c6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:13.936139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 111/207] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:13 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:13.942247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:14.013997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:14.089301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6101-c326-4b70-9a05-064e84e52450 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:14.139145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:14.139679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.139959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.141042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.141803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] RBAC: Adding query filter params (role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0, scope.project.id=c63e8d69313f433695ce236bb79eba17, user.id=ca23ea22e99a4c88acff38ef4ed51102) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:14.142111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:14.143548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ddb14-8edf-4bea-840a-e8ff88ca6cd4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:14.152745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 97/208] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:13 2022] GET /identity/v3/role_assignments?user.id=ca23ea22e99a4c88acff38ef4ed51102&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:14.241190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dadd6c-3e86-42cc-8acd-0cf6b97ef623 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.241563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dadd6c-3e86-42cc-8acd-0cf6b97ef623 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.241782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dadd6c-3e86-42cc-8acd-0cf6b97ef623 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.243280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 112/209] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:14.249125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b39b7-2054-4874-b858-90e627a51572 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.249752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b39b7-2054-4874-b858-90e627a51572 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.250376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b39b7-2054-4874-b858-90e627a51572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.278134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-098b39b7-2054-4874-b858-90e627a51572 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:14.280269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-098b39b7-2054-4874-b858-90e627a51572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:14.318616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6461-c8fc-441e-85f6-9458ca624691 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.319143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6461-c8fc-441e-85f6-9458ca624691 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.319143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6461-c8fc-441e-85f6-9458ca624691 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.320844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 113/210] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:14.326490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.326778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.327065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.365503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:14.365503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:14.382053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 98/211] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:14.464721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 114/212] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:14.676951 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b39b7-2054-4874-b858-90e627a51572 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:14.712896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:14.770880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:14.790711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b39b7-2054-4874-b858-90e627a51572 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:14.835561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:14.840936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:14.841633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.841915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.842192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.855716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:14.862791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:14.863688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:14.865449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 99/213] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:14.869923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:14.921227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b24e-1a4c-4d72-b6e7-1753b6f2561c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:14.960276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:14.961937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:14.962403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:14.962864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:14.964158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:14.966271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:14.972865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 115/214] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:14.975661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:15.009532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:15.020050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081ab4c-3be0-4694-a31b-7718ca58e8f7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:15.067330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:15.068492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.068935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.069355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.070369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:15.070876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:15.073175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:15.089658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 100/215] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:15.090991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:15.113603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bec65e-6234-435f-8c11-dabc475b3078 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:15.137740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:15.160952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:15.161623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.161931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.162211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.162870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:15.164820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:15.197567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 116/216] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:14 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:15.285766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005160f-bc59-4d6b-9719-38a4f27d7407 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:15.381157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbede0f6-e164-4aa3-a215-1aa839dfd2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:15.382331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede0f6-e164-4aa3-a215-1aa839dfd2ee None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.382896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede0f6-e164-4aa3-a215-1aa839dfd2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.383387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede0f6-e164-4aa3-a215-1aa839dfd2ee None admin] PATH_INFO: `/v3/users/a4aa493faae54edbb7edf873f12bb433` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.403902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede0f6-e164-4aa3-a215-1aa839dfd2ee None admin] RBAC: Authorizing `identity:get_user(user_id=a4aa493faae54edbb7edf873f12bb433)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:15.405837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede0f6-e164-4aa3-a215-1aa839dfd2ee None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:15.408395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 101/217] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:15 2022] GET /identity/v3/users/a4aa493faae54edbb7edf873f12bb433 => generated 297 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:15.414307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:15.445033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50079b9-80e8-458b-b42f-b416f9c95927 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.445686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50079b9-80e8-458b-b42f-b416f9c95927 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.446142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50079b9-80e8-458b-b42f-b416f9c95927 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.447746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 102/218] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:15.459820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af56c-a486-4039-9975-ba4303127464 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.460500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af56c-a486-4039-9975-ba4303127464 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.461009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af56c-a486-4039-9975-ba4303127464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.491290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0c0af56c-a486-4039-9975-ba4303127464 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:15.492958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0c0af56c-a486-4039-9975-ba4303127464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:15.512240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:15.582971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708138c9-dba7-43b6-8f06-b04c48a117c9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:15.604621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 103/219] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:15.606059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb58ed-ebfe-4445-a931-598a7d73c12d None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.606336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb58ed-ebfe-4445-a931-598a7d73c12d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.606786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb58ed-ebfe-4445-a931-598a7d73c12d None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.608492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 104/220] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:15.615736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.616023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.616309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.632974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:15.633742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:15.634118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:15.635130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:15.639764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:15.641765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:15.721924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:15.738537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 105/221] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:15.765999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:15.767683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:15.887773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 117/222] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:15.898723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.023193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.023959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.078813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b764de3-8bd2-42ef-8d19-a94c55367d1d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.132442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.134935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.135226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.135508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.137070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:16.138788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:16.141493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 106/223] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:15 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:16.142376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.152195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.179862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50651c86-8d59-4a66-93ed-dab278a34694 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.217957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.218807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.219180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.219621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.230640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:16.236527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:16.237390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:16.238710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 118/224] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:16.243870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.255925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ca6de-8735-4246-9477-44676d4e8069 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.293409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.294240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.294527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.297483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.306819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:16.312543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:16.314628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:16.315729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 107/225] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:16.321483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.324878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.362326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42586646-f444-489c-a27d-c93cb54c4c8c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.400068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.401065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.401693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.402168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.403097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=a4aa493faae54edbb7edf873f12bb433) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:16.403607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:16.405621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:16.412073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 119/226] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/role_assignments?user.id=a4aa493faae54edbb7edf873f12bb433&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:16.413196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.417909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.473643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858504f0-83b7-4d9b-8c30-4263a1b816ee None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.561682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.563480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.564064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.564555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.565436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:16.569075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:16.570975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:16.583982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 108/227] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:16.590933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.628412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f0bb0-e80c-423d-8e82-1747bd87d8d4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.664519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.672704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.673394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.673668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.674019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.674647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:16.674951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:16.676386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:16.686153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 120/228] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:16.692513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.715561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6b2ea-fdfb-49c1-9af4-627713bd36aa None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.755508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c34332-e187-4203-a883-550517e160fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.756662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c34332-e187-4203-a883-550517e160fc None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.757268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c34332-e187-4203-a883-550517e160fc None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.757773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c34332-e187-4203-a883-550517e160fc None admin] PATH_INFO: `/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.759114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c34332-e187-4203-a883-550517e160fc None admin] RBAC: Authorizing `identity:get_user(user_id=d64cd8d1f3ed4965859cd2f53b6497a0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:16.760856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c34332-e187-4203-a883-550517e160fc None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:16.762693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 109/229] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:16.769988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c34332-e187-4203-a883-550517e160fc None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:16.825843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.881769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c34332-e187-4203-a883-550517e160fc None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.913799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:16.919711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:16.920311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:16.920595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:16.920872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:16.923051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:16.923805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1909c1-30fc-4ad6-b95d-e21f8e1432ab None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:16.924866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:16.931324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 110/230] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:16.932587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:17.001593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:17.002247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:17.002531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:17.002799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] PATH_INFO: `/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:17.004805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] RBAC: Authorizing `identity:get_user(user_id=d64cd8d1f3ed4965859cd2f53b6497a0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:17.006316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:17.007907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 121/231] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:16 2022] GET /identity/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0 => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:17.013881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:17.084781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af4bfb-2a4c-45da-aa25-494df55a6aed None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:17.151879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852dc03-6ebb-411f-9c51-8fb3d4244b2e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:17.175228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:17.177114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:17.177618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:17.179085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/d64cd8d1f3ed4965859cd2f53b6497a0/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:17.182074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c, user_id=d64cd8d1f3ed4965859cd2f53b6497a0, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:17.185191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:17.196967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:17.201255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4245c8-cd3e-4a15-b6c5-12839c06e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:17.202228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4245c8-cd3e-4a15-b6c5-12839c06e26b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:17.202729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4245c8-cd3e-4a15-b6c5-12839c06e26b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:17.206884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4245c8-cd3e-4a15-b6c5-12839c06e26b None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:17.211129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 111/232] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:16 2022] PUT /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/d64cd8d1f3ed4965859cd2f53b6497a0/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:17.212494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4245c8-cd3e-4a15-b6c5-12839c06e26b None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:17.214318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4245c8-cd3e-4a15-b6c5-12839c06e26b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:17.216677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 122/233] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:17 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:17.223127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:17.300039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:17.354732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536dd5d-f601-4e10-981b-870dc9173c7c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:17.402690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f7833-e7f3-429e-8779-2209a0568d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:17.403710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7833-e7f3-429e-8779-2209a0568d95 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:17.404239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7833-e7f3-429e-8779-2209a0568d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:17.404692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7833-e7f3-429e-8779-2209a0568d95 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/d64cd8d1f3ed4965859cd2f53b6497a0/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:17.408179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f7833-e7f3-429e-8779-2209a0568d95 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c, user_id=d64cd8d1f3ed4965859cd2f53b6497a0, role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:17.409928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f7833-e7f3-429e-8779-2209a0568d95 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:17.438553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 112/234] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:17 2022] PUT /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/d64cd8d1f3ed4965859cd2f53b6497a0/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:17.813047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdbd5-1528-4009-88b7-f9bece689899 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:17.813914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdbd5-1528-4009-88b7-f9bece689899 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:17.814192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdbd5-1528-4009-88b7-f9bece689899 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:17.815688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 123/235] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:17.821818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c3201-48b6-4be8-b673-eacdbaa27250 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:17.822369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c3201-48b6-4be8-b673-eacdbaa27250 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:17.822844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c3201-48b6-4be8-b673-eacdbaa27250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:17.842153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-3b2c3201-48b6-4be8-b673-eacdbaa27250 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:17.843737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3b2c3201-48b6-4be8-b673-eacdbaa27250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:17.934731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 113/236] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:18.048626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a84c-2dfb-4a58-a752-e6611d23fc37 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:18.048943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a84c-2dfb-4a58-a752-e6611d23fc37 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:18.049175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a84c-2dfb-4a58-a752-e6611d23fc37 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:18.050707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 124/237] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:18.056142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:18.056428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:18.056706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:18.084508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:18.085658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:18.147323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:18.171754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 114/238] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:18.405707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a84c-2dfb-4a58-a752-e6611d23fc37 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:18.470163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:18.567639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a84c-2dfb-4a58-a752-e6611d23fc37 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:18.634929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c056c-0b27-41bb-9cd8-20e0e356ade7 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:18.647858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:18.648501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:18.648779 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:18.649057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:18.665880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:18.677287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:18.678214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:18.678993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:18.680115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 125/239] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:18.686995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:18.707439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:18.709005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:18.709289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:18.710091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:18.711397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:18.713811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:18.716849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 115/240] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:18.722757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:18.787374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:18.820491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:18.849568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef5ab-c4af-4c25-8162-fc7ee1875410 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:18.905062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:18.905738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:18.906576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:18.907015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:18.907712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:18.908017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:18.914343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:18.934689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 126/241] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:18.941372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:18.956417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb154-4a17-44a4-8113-330189ef601b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:19.031650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd6b6-0b8c-44d7-9af6-06eef3bf56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:19.032277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd6b6-0b8c-44d7-9af6-06eef3bf56db None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.032563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd6b6-0b8c-44d7-9af6-06eef3bf56db None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.032843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd6b6-0b8c-44d7-9af6-06eef3bf56db None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.033528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd6b6-0b8c-44d7-9af6-06eef3bf56db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:19.035099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd6b6-0b8c-44d7-9af6-06eef3bf56db None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:19.057956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14793d4e-38f2-4bae-845f-f27a838690a4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:19.062940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 116/242] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:18 2022] POST /identity/v3/users => generated 293 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:19.064362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85a72-6ace-439b-9d5a-c23f4b10ee09 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.064937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85a72-6ace-439b-9d5a-c23f4b10ee09 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.065271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85a72-6ace-439b-9d5a-c23f4b10ee09 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.066957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 117/243] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:19.075045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.075652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.076153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.117392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:19.117882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:19.119593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:19.131292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:19.134244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.134709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.135711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] PATH_INFO: `/v3/users/648606f46ed54d869566cbe90451896c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.170179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] RBAC: Authorizing `identity:get_user(user_id=648606f46ed54d869566cbe90451896c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:19.173429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:19.175724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 127/244] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:18 2022] GET /identity/v3/users/648606f46ed54d869566cbe90451896c => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:19.183910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:19.254418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 118/245] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:19.319018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549ae9-ba7b-4c86-8a45-8d7379882b64 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:19.353933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:19.354860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.355354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.355834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.404557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:19.432061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:19.433565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:19.547347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 128/246] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:19.553903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:19.591901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:19.595966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:19.694662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847d7d6-3e90-42dc-8f3b-ff1a42665a90 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:19.710214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:19.740433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:19.741617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.742108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.742905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.744611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:19.745310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48629b1-d96f-415d-b983-6a34082ddb55 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:19.746488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:19.750464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 119/247] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:19 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:19.757946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:19.804491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:19.805046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.805324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.805642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.820147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:19.825917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:19.826896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:19.827688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 129/248] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:19.834640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:19.919868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90787bbc-aa60-4b80-a050-0ff8691c133a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:19.921382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:19.969546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:19.970442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:19.970866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:19.971283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:19.972165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] RBAC: Adding query filter params (role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=648606f46ed54d869566cbe90451896c) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:19.973112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:19.975074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:19.982806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 120/249] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:19 2022] GET /identity/v3/role_assignments?user.id=648606f46ed54d869566cbe90451896c&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:19.992638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2dfc0-43b1-4a0c-b143-80b0db6e850a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:20.045188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:20.046155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.046587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.047051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.047915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:20.048374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:20.050075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22419564-4eb4-4792-9b32-d2d58e23ad22 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:20.053248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:20.065617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 130/250] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:20.067690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:20.151337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e4184-4874-4e6c-8613-940177426b66 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.151630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e4184-4874-4e6c-8613-940177426b66 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.151914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e4184-4874-4e6c-8613-940177426b66 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.154660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 131/251] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:20.159022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7434-bda5-4abf-a837-dfeeec082311 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.159319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7434-bda5-4abf-a837-dfeeec082311 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.159603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7434-bda5-4abf-a837-dfeeec082311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.178637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d06c7434-bda5-4abf-a837-dfeeec082311 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:20.179854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d06c7434-bda5-4abf-a837-dfeeec082311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:20.205312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a408dc-5ac5-45d7-8c92-5756f6108f68 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:20.287649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:20.288304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.288591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.288863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] PATH_INFO: `/v3/users/a4aa493faae54edbb7edf873f12bb433` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.291426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] RBAC: Authorizing `identity:get_user(user_id=a4aa493faae54edbb7edf873f12bb433)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:20.291681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:20.293687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 121/252] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:20 2022] GET /identity/v3/users/a4aa493faae54edbb7edf873f12bb433 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:20.299560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:20.306630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 132/253] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:20.355003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:20.410766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8641800-7616-4d6a-9820-de3b95a1563a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.411095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8641800-7616-4d6a-9820-de3b95a1563a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.411309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8641800-7616-4d6a-9820-de3b95a1563a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.412828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 133/254] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:20.415007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0837c-0e9c-449e-937a-e4ab2dfac34a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:20.417903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.418182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.418452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.457745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:20.458943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:20.485525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:20.486184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.486471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.486749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.489051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:20.490785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:20.492967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 122/255] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:20 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:20.501414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:20.617455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 134/256] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 10 20:46:20.691306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950de6-a8f4-4f95-bdd7-64b7de92e75a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:20.693497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:20.741907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:20.742954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:20.743460 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:20.743933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/a4aa493faae54edbb7edf873f12bb433/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:20.747935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] RBAC: Authorizing `identity:create_grant(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c, user_id=a4aa493faae54edbb7edf873f12bb433, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:20.749630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:20.785782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 123/257] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:20 2022] PUT /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/a4aa493faae54edbb7edf873f12bb433/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:20.797664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:20.887190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:20.962204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd22bb1-686f-424e-8663-efeac1eae797 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:21.009723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:21.010372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:21.010625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:21.010896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:21.021974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:21.026767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:21.027621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:21.028411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 135/258] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:21.035911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.150453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.196710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6090e60-9660-4665-ac47-500eb9a9970b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:21.264551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:21.265878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:21.266359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:21.266826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:21.267706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:21.268223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:21.278004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:21.288981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 124/259] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:21.299325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.342004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:21.373490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690cdc3-05a8-4bc3-a203-6b393ff03936 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:21.417919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:21.445610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:21.446312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:21.446537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:21.446821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:21.448994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c782e-3d52-42ad-b307-e518d058b59c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:21.470561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:21.475765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:21.476639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:21.477429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:21.484788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 136/260] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:21.484788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.495066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:21.496080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:21.496553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:21.497020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] PATH_INFO: `/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:21.498413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] RBAC: Authorizing `identity:get_user(user_id=d64cd8d1f3ed4965859cd2f53b6497a0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:21.500736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:21.502916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 125/261] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:21.514693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.627007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:21.641490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61635-dbac-4c0d-9480-5679d59d9caf None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:21.676137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:21.676684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:21.676965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:21.677295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:21.711416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce9e64-dc1b-4bec-a505-8d1e5e85dd6f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:21.750435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:21.752397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:21.826796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:21.827439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:21.827724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:21.828088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:21.828769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:21.829074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:21.830511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:21.847584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 137/262] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:21.848246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:21.857298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 126/263] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:46:21.858499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.863283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:21.960059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:22.010982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7833e1f-b1b2-4131-b870-c5f0e4593870 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:22.054185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:22.054828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.055109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.055383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:22.056642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:22.058123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:22.060343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 127/264] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:22.067076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:22.096877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc0b1a-b2e9-4afa-84ee-3d5f1a47977c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:22.158238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:22.158942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.159230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.159507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] PATH_INFO: `/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:22.160684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=d64cd8d1f3ed4965859cd2f53b6497a0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:22.162187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:22.163795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 138/265] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:21 2022] GET /identity/v3/users/d64cd8d1f3ed4965859cd2f53b6497a0 => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:22.170168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:22.194606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:22.254631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992167-69c8-48b5-9080-ed2196a97a65 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:22.322446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:22.323088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.323542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.323816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:22.324564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=d64cd8d1f3ed4965859cd2f53b6497a0) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:22.324871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:22.326350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:22.340255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 128/266] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:22 2022] GET /identity/v3/role_assignments?user.id=d64cd8d1f3ed4965859cd2f53b6497a0&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:22.377737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30828c76-16dd-48c9-bd68-69b0eb306ce3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:22.393798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:22.423770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:22.424402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.424687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.424959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:22.529731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:22.531880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:22.644727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 139/267] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:46:22.650214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:22.796867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a75b2f-d33d-478c-935c-a75f226a6e11 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:22.829558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:22.845477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4fc57-b1ae-4073-85d9-5155d2ff8c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:22.846493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4fc57-b1ae-4073-85d9-5155d2ff8c25 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.847003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4fc57-b1ae-4073-85d9-5155d2ff8c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.847478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4fc57-b1ae-4073-85d9-5155d2ff8c25 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:22.854176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4fc57-b1ae-4073-85d9-5155d2ff8c25 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:22.855900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4fc57-b1ae-4073-85d9-5155d2ff8c25 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:22.858170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 129/268] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:22 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:22.863798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:22.884506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:22.979839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eb8ee-13ba-4dcd-8d83-150483a72d43 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.980494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eb8ee-13ba-4dcd-8d83-150483a72d43 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.981003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eb8ee-13ba-4dcd-8d83-150483a72d43 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:22.982859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 130/269] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:22.988734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab45b83-924d-4e00-bb1f-7c026b4b7766 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:22.989294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab45b83-924d-4e00-bb1f-7c026b4b7766 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:22.989814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab45b83-924d-4e00-bb1f-7c026b4b7766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.020074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bab45b83-924d-4e00-bb1f-7c026b4b7766 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:23.022368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6ae2c-5e6c-4fc8-8e2e-8917b1a4fa72 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:23.022908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bab45b83-924d-4e00-bb1f-7c026b4b7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:23.085634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:23.087483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.087928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.088208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.088948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] RBAC: Adding query filter params (role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=d64cd8d1f3ed4965859cd2f53b6497a0) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:23.089260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:23.090760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4a234-c962-465f-8960-3d38d345297c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:23.101504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 140/270] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:22 2022] GET /identity/v3/role_assignments?user.id=d64cd8d1f3ed4965859cd2f53b6497a0&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:23.101504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542518d-ab56-4d66-b35f-e83a54172a33 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.101504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542518d-ab56-4d66-b35f-e83a54172a33 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.101504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542518d-ab56-4d66-b35f-e83a54172a33 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.104133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 141/271] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:23.106653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.107162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.107280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.135818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:23.137401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:23.160589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 131/272] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:23.166410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321eea2-5cf5-4987-9a5d-0c9971a68848 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.167019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321eea2-5cf5-4987-9a5d-0c9971a68848 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.167513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321eea2-5cf5-4987-9a5d-0c9971a68848 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.169443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 132/273] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:23.175198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.175754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.176257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.210611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:23.212190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:23.232551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 142/274] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:23.332726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 133/275] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:23.476357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:23.479368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:23.547998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:23.588502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16865436-2ba1-4520-95c5-6cebcb2eaeec None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:23.628357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:23.629092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.629378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.629768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.631367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:23.632822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:23.633246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:23.635353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 143/276] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:23.641629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:23.691530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1574e-d9e3-42dd-8c56-eeb48c75e9cd None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:23.719139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:23.729601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:23.730317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.730623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.730920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.741792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:23.746663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:23.747544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:23.748354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 134/277] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:23.749019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:23.761409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be0b2-4b58-431c-a69f-5973ff3b5ebd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:23.801046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:23.801740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.802016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.802286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.813216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:23.818148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:23.819016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 144/278] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:23.820701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:23.858177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3fabb-8829-4987-bb60-88bac60640cb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:23.872840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:23.897870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:23.898955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.898955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.899132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.899666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:23.901111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:23.924089 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 135/279] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:23 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:23.924089 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:23.933860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a432cd-5a25-4da7-b618-b07fe51979f6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:23.972693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:23.980958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:23.981558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:23.981888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:23.982170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:23.982852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:23.983160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:23.984695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.000508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 145/280] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:24.002406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:24.056888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef326d0a-125d-4de0-82c7-dc2f1fdfb146 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.105095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.105903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.106183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.106458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.107150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:24.107453 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:24.109133 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e0a45-27a9-4531-953b-87cf8841194e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.118690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 136/281] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:24.120088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21b48f-4435-4c7f-a121-aa33c2532b0e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.120371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21b48f-4435-4c7f-a121-aa33c2532b0e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.120641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21b48f-4435-4c7f-a121-aa33c2532b0e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.121962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 137/282] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:24.129328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21b48f-4435-4c7f-a121-aa33c2532b0e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:24.131642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616f41-ff31-4711-80cc-9c672ec39ef3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.166404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:24.172730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb0462-c2fa-4ebc-a5d7-2feea251a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.173372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb0462-c2fa-4ebc-a5d7-2feea251a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.173724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb0462-c2fa-4ebc-a5d7-2feea251a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.174003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb0462-c2fa-4ebc-a5d7-2feea251a1e7 None admin] PATH_INFO: `/v3/users/b38769d1ad5d4b2c94337728683a9ff9` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.189985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb0462-c2fa-4ebc-a5d7-2feea251a1e7 None admin] RBAC: Authorizing `identity:get_user(user_id=b38769d1ad5d4b2c94337728683a9ff9)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:24.191467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb0462-c2fa-4ebc-a5d7-2feea251a1e7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.193133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 146/283] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:24 2022] GET /identity/v3/users/b38769d1ad5d4b2c94337728683a9ff9 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:24.195274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.195558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.195831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.214459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:24.215661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:24.264688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21b48f-4435-4c7f-a121-aa33c2532b0e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.272756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:24.298720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 147/284] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:24.299921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:24.320090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.320741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.321021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.321291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] PATH_INFO: `/v3/users/648606f46ed54d869566cbe90451896c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.322621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] RBAC: Authorizing `identity:get_user(user_id=648606f46ed54d869566cbe90451896c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:24.324138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.326700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 138/285] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:24 2022] GET /identity/v3/users/648606f46ed54d869566cbe90451896c => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:24.332596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:24.423357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca795c-e7c8-405b-be58-177df213c198 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.477836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:24.481106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.481754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.482035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.482336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.483040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, user.id=b38769d1ad5d4b2c94337728683a9ff9, scope.system=all) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:24.483351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:24.487236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c33e8-a1dd-4edc-a36e-5bea19d4683c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.489170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.500082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 148/286] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Sep 10 20:46:24 2022] GET /identity/v3/role_assignments?user.id=b38769d1ad5d4b2c94337728683a9ff9&scope.system=all&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:24.529015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.529730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.530020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.530348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.531708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:24.533514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.535601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 139/287] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:24 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:24.541940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:24.624314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:24.708000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35ddc4-992c-4080-a93c-e092145db2fc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.737450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:24.754959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.755719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.755996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.756265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/648606f46ed54d869566cbe90451896c/roles/d20f3e89a1aa4fa58b9d6cedc13ed5f0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.759010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] RBAC: Authorizing `identity:create_grant(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c, user_id=648606f46ed54d869566cbe90451896c, role_id=d20f3e89a1aa4fa58b9d6cedc13ed5f0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:24.760565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:24.780443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 149/288] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:46:24 2022] PUT /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/users/648606f46ed54d869566cbe90451896c/roles/d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:24.871736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b53cd2-cedf-4b2e-954e-98673aeb51ef None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:24.912821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:24.913776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:24.914201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:24.914646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:24.927257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:24.933195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:24.934338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:24.935320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:24.938248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 140/289] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:24.944325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:25.073193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f173-db56-4f79-8b48-a0419656711d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:25.075980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:25.114778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:25.115712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.116206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.116677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.117602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:25.118112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:25.119658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:25.128834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 150/290] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:25.135005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:25.168761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:25.254235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9568da-2e4b-4e1f-b9fc-4fab0e1db4c5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:25.294953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:25.295593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.295873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.296150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] PATH_INFO: `/v3/users/a4aa493faae54edbb7edf873f12bb433` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.298154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] RBAC: Authorizing `identity:get_user(user_id=a4aa493faae54edbb7edf873f12bb433)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:25.299620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:25.301634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 141/291] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:25 2022] GET /identity/v3/users/a4aa493faae54edbb7edf873f12bb433 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:25.307311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:25.435224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00cb13-3a56-446a-8176-2781b6a2a6bf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:25.476344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:25.476960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.477240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.477550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.519501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:25.563781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:25.565300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:25.705147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 151/292] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:25.711020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:25.851376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:25.861260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5a0cf-495a-4f2a-8215-dd0ab0671f4c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:25.910090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b65cd0-76ee-417e-87b6-d9cb869fcfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:25.910793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b65cd0-76ee-417e-87b6-d9cb869fcfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.911084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b65cd0-76ee-417e-87b6-d9cb869fcfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.911366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b65cd0-76ee-417e-87b6-d9cb869fcfe2 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.912577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b65cd0-76ee-417e-87b6-d9cb869fcfe2 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:25.914126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b65cd0-76ee-417e-87b6-d9cb869fcfe2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:25.916097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 142/293] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:25 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:25.922820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:26.035909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:26.088347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f73df8-7e49-419c-8406-049e7ef4e8f0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:26.135129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:26.135734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:26.136012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:26.136283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:26.137008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=a4aa493faae54edbb7edf873f12bb433) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:26.137309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:26.138765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b7394-5e35-4a36-b921-6cad6f3bb28c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:26.145328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 152/294] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:25 2022] GET /identity/v3/role_assignments?user.id=a4aa493faae54edbb7edf873f12bb433&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:26.822831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751febb5-4aec-4bc9-b888-85522d121ac5 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:26.822831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751febb5-4aec-4bc9-b888-85522d121ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:26.822831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751febb5-4aec-4bc9-b888-85522d121ac5 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:26.822831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 143/295] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:26.829087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:26.829986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:26.830454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:26.849911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:26.851419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:26.870718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e19e-bbc6-4895-a643-118f2eac3a02 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:26.871330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e19e-bbc6-4895-a643-118f2eac3a02 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:26.871795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e19e-bbc6-4895-a643-118f2eac3a02 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:26.873599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 144/296] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:46:26.892261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee92a-971a-45e7-826c-d6cbb81e30e4 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:26.892768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee92a-971a-45e7-826c-d6cbb81e30e4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:26.893285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee92a-971a-45e7-826c-d6cbb81e30e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:26.915499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f65ee92a-971a-45e7-826c-d6cbb81e30e4 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:26.917436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f65ee92a-971a-45e7-826c-d6cbb81e30e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:26.938293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 153/297] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:27.000055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 145/298] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:27.200909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.261854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:27.268689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4ac4a-b85c-471d-bbb2-78c2847f844c None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.269301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4ac4a-b85c-471d-bbb2-78c2847f844c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.269804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4ac4a-b85c-471d-bbb2-78c2847f844c None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.271537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 146/299] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:27.277085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.277085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.277085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.296953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:27.298268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:27.304983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bd0e4-22d2-477d-9568-7fc74e9e549c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.346221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.347453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.347740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.348017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.360849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:27.367888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:27.368775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:27.369598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:27.370193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 154/300] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:27.373498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.375601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 147/301] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:27.376859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.406631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:27.495738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbd269-6dc0-40e9-b8ab-d54b20a94f63 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.498525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da430ac-4134-4db2-aa0e-59f5bb09a2ef None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.520414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:27.533689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.534485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.534774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.534991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fd0a1-852a-41ff-88dd-4718a59cc170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.535120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.535619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd0a1-852a-41ff-88dd-4718a59cc170 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.535715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:27.535897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd0a1-852a-41ff-88dd-4718a59cc170 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.535990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:27.536169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd0a1-852a-41ff-88dd-4718a59cc170 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.537275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fd0a1-852a-41ff-88dd-4718a59cc170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:27.537443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45950718-5540-4dc4-93e8-5051bcdb4520 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:27.538756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fd0a1-852a-41ff-88dd-4718a59cc170 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:27.540521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 155/302] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:27.543553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5fb94-b41a-4d98-b0ce-86cdd332421f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.544047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5fb94-b41a-4d98-b0ce-86cdd332421f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.544821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5fb94-b41a-4d98-b0ce-86cdd332421f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.546244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 156/303] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:27.547154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 148/304] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:27.552560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5fb94-b41a-4d98-b0ce-86cdd332421f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.552899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.553158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.553435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.571072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:27.572248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:27.648918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 149/305] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:27.649869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.665812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:27.668270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5fb94-b41a-4d98-b0ce-86cdd332421f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.706358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.707365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.707836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.708313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] PATH_INFO: `/v3/groups` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.709005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:27.710470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:27.741705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 157/306] 10.222.0.23 () {60 vars in 1134 bytes} [Sat Sep 10 20:46:27 2022] POST /identity/v3/groups => generated 229 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:27.743119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.762487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b3c58-c0c2-45c2-8406-ce6422782d8a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.813845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:27.820636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.821483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.821918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.822553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] PATH_INFO: `/v3/users/beb5e5060d73438886c5cd89ae6aee81` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.845206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] RBAC: Authorizing `identity:get_user(user_id=beb5e5060d73438886c5cd89ae6aee81)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:27.846846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:27.849039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 150/307] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/users/beb5e5060d73438886c5cd89ae6aee81 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:27.854670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.877875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c97af5-c64d-4013-8da7-6d29d5356e9c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.928176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.928821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.929094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.929366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.941922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:27.947341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:27.948257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:27.949057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 158/308] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:27.949733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.958654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:27.994714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4f2f7-de70-4826-a085-a5ab42950e5d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.034384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.035009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.035293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.035567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.036294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] RBAC: Adding query filter params (role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0, user.id=beb5e5060d73438886c5cd89ae6aee81, scope.system=all) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:28.036725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.038185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.043649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 151/309] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/role_assignments?user.id=beb5e5060d73438886c5cd89ae6aee81&scope.system=all&role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:28.044285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:28.081114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad2ae6-433e-4de7-a2ea-7552324de5e0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.095948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:28.131477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.132307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.132688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.133063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.146101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:28.151925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:28.153054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:28.154059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:28.155857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 159/310] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:28.161709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:28.184577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4f02b-d75e-44f8-81d9-3d448ea15e28 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.233890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.235390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.235975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.236390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.237768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:28.238239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.240023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.243649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:28.253046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 152/311] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:28.259143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:28.275863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012aef-f0b1-4a56-809c-e0841c9b0234 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.314661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.315214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.315488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.315775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.316418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:28.316715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.318331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.330387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 160/312] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:28.335120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:28.371120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:28.381422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e174e76-82bb-4895-af7b-35e51feab274 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.424377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.425261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.425725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.426146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] PATH_INFO: `/v3/users/b38769d1ad5d4b2c94337728683a9ff9` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.427470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] RBAC: Authorizing `identity:get_user(user_id=b38769d1ad5d4b2c94337728683a9ff9)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.429066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.430827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 153/313] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:28 2022] GET /identity/v3/users/b38769d1ad5d4b2c94337728683a9ff9 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:28.436474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:28.466854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc87833-6a3b-456c-a514-9584ecc3d24a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.512941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:28.519221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.519871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.520152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.520428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] PATH_INFO: `/v3/users/648606f46ed54d869566cbe90451896c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.522327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] RBAC: Authorizing `identity:get_user(user_id=648606f46ed54d869566cbe90451896c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.523968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.530152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 161/314] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:28 2022] GET /identity/v3/users/648606f46ed54d869566cbe90451896c => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:28.531528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:28.561718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777462-bbd0-40d0-a544-aeb006c6a51f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.598441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.599399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.599516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.599730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] PATH_INFO: `/v3/system/users/b38769d1ad5d4b2c94337728683a9ff9/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.601153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b38769d1ad5d4b2c94337728683a9ff9, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.602620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.613409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 154/315] 10.222.0.23 () {58 vars in 1334 bytes} [Sat Sep 10 20:46:28 2022] PUT /identity/v3/system/users/b38769d1ad5d4b2c94337728683a9ff9/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:46:28.668596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:28.674283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acc4d-cee6-42ba-a756-83f0a594865f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:28.717718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:28.719691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:28.720287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:28.720738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:28.803548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:28.805007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:28.918736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:28.924453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 162/316] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:28.930082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:29.041745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c96a5-c22f-46ec-b576-3b8af22485d5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:29.073136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:29.080879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615818e6-cfd5-4ae9-bad8-4ed086a6a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:29.081810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615818e6-cfd5-4ae9-bad8-4ed086a6a46e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:29.082317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615818e6-cfd5-4ae9-bad8-4ed086a6a46e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:29.082789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615818e6-cfd5-4ae9-bad8-4ed086a6a46e None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:29.084393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615818e6-cfd5-4ae9-bad8-4ed086a6a46e None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:29.086109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615818e6-cfd5-4ae9-bad8-4ed086a6a46e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:29.088271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 155/317] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:28 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:29.095017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:29.204601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c3a6c-426a-4795-819b-b67a7f3e6b8e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:29.242902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:29.243935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:29.244410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:29.244946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:29.245894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] RBAC: Adding query filter params (role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c, user.id=648606f46ed54d869566cbe90451896c) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:29.246419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:29.248083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e4b6-be18-403e-8471-d79f2a5b9c4d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:29.255770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 163/318] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:46:29 2022] GET /identity/v3/role_assignments?user.id=648606f46ed54d869566cbe90451896c&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:30.041141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16636c-344a-42f0-9a54-b04b7fbb1fa5 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.041586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16636c-344a-42f0-9a54-b04b7fbb1fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.041787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16636c-344a-42f0-9a54-b04b7fbb1fa5 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.043450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 156/319] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:30.048694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470daab-254e-4510-9deb-41c40df46023 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.049076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470daab-254e-4510-9deb-41c40df46023 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.049234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470daab-254e-4510-9deb-41c40df46023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.067678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9470daab-254e-4510-9deb-41c40df46023 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:30.068860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9470daab-254e-4510-9deb-41c40df46023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:30.126615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:30.137432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 164/320] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:30.276321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaddfdd-d7d9-470f-b3e2-a41348a83036 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.276626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaddfdd-d7d9-470f-b3e2-a41348a83036 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.276890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaddfdd-d7d9-470f-b3e2-a41348a83036 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.278408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 157/321] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:30.284300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.284596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.284875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.303160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:30.304339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:30.372067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 165/322] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:30.406365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaddfdd-d7d9-470f-b3e2-a41348a83036 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:30.513916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaddfdd-d7d9-470f-b3e2-a41348a83036 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:30.547580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:30.548144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.548428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.548703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.550033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:30.551420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:30.553174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 158/323] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:30.558742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:30.643142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:30.661141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd18a-38c8-4540-802b-1ab39967f01e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:30.685188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:30.698208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:30.698918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.699097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.699371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] PATH_INFO: `/v3/groups` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.700020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:30.701486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:30.717293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:30.717597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 166/324] 10.222.0.23 () {60 vars in 1134 bytes} [Sat Sep 10 20:46:30 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:46:30.758481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a173d7a-7d64-4377-b184-bb8bd72c894e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:30.810704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e46e0-9365-4bf0-9086-55e124c2a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:30.811263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e46e0-9365-4bf0-9086-55e124c2a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.811543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e46e0-9365-4bf0-9086-55e124c2a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.811816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e46e0-9365-4bf0-9086-55e124c2a5a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.824707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e46e0-9365-4bf0-9086-55e124c2a5a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-0c6e46e0-9365-4bf0-9086-55e124c2a5a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:30.830307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:30.831254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:30.832109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 159/325] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:30.837439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:30.899388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed982cff-da0e-40a9-977c-fbf68b34b424 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.899674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed982cff-da0e-40a9-977c-fbf68b34b424 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.899948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed982cff-da0e-40a9-977c-fbf68b34b424 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.901303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 160/326] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:46:30.906929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:30.907492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:30.907776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:30.931010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:30.931959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:30.969423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8473a2-427a-4c14-89ef-fc91517b161d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:31.008294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 161/327] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:31.021189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:31.021846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.022200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.022448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.023100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:31.023410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:31.024849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:31.034427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 167/328] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:31.040003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:31.104125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:31.153508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb6a5e-8453-46ac-83bd-8c88ea119ded None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:31.192357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:31.193001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.193282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.193588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] PATH_INFO: `/v3/users/beb5e5060d73438886c5cd89ae6aee81` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.194716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] RBAC: Authorizing `identity:get_user(user_id=beb5e5060d73438886c5cd89ae6aee81)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:31.196176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:31.198163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 162/329] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:31 2022] GET /identity/v3/users/beb5e5060d73438886c5cd89ae6aee81 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:31.204383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:31.255575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:31.310805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:31.328980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86990e79-9c92-4a90-b091-52378fd977b3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:31.383259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:31.384153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.384546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.384826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] PATH_INFO: `/v3/system/users/beb5e5060d73438886c5cd89ae6aee81/roles/d20f3e89a1aa4fa58b9d6cedc13ed5f0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.386413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=beb5e5060d73438886c5cd89ae6aee81, role_id=d20f3e89a1aa4fa58b9d6cedc13ed5f0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:31.388023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:31.404970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 168/330] 10.222.0.23 () {58 vars in 1334 bytes} [Sat Sep 10 20:46:31 2022] PUT /identity/v3/system/users/beb5e5060d73438886c5cd89ae6aee81/roles/d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:46:31.442891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b60f-550f-4475-a2fb-4d9673ec80cd None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:31.471364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:31.484576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:31.485931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.486213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.486485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.497793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:31.503476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:31.504319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:31.505110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 163/331] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:31.510642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:31.642125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5df37c-8bac-49ee-835d-45b6b60bc30a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:31.652525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:31.684783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:31.685405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.685713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.685993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.686663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:31.686968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:31.688571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:31.701713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 169/332] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:31.708018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:31.824565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b9b7-e679-473c-9294-7336560752bd None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:31.861547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a0e6a-5b2f-4706-a04f-574e173fdae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:31.862560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a0e6a-5b2f-4706-a04f-574e173fdae2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.863031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a0e6a-5b2f-4706-a04f-574e173fdae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.863494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a0e6a-5b2f-4706-a04f-574e173fdae2 None admin] PATH_INFO: `/v3/users/b38769d1ad5d4b2c94337728683a9ff9` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.864845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a0e6a-5b2f-4706-a04f-574e173fdae2 None admin] RBAC: Authorizing `identity:get_user(user_id=b38769d1ad5d4b2c94337728683a9ff9)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:31.866499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a0e6a-5b2f-4706-a04f-574e173fdae2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:31.868563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 164/333] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:31 2022] GET /identity/v3/users/b38769d1ad5d4b2c94337728683a9ff9 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:31.874048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:32.035315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ea4d-7157-4710-aa33-7660af019792 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:32.082349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:32.082967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:32.083243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:32.083514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:32.084294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] RBAC: Adding query filter params (role.id=3e2944846ed042f0bf3a6b134cb87695, user.id=b38769d1ad5d4b2c94337728683a9ff9, scope.system=all) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:32.084608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:32.086080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58048bc3-4f26-4441-ae44-161bf9173f8d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:32.103826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 170/334] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Sep 10 20:46:31 2022] GET /identity/v3/role_assignments?user.id=b38769d1ad5d4b2c94337728683a9ff9&scope.system=all&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:34.099456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d4c13-accc-4f06-8b39-403812a49f76 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.100004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d4c13-accc-4f06-8b39-403812a49f76 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.100004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d4c13-accc-4f06-8b39-403812a49f76 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.101672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 165/335] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:34.107520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205765-664f-4874-87e9-1f90e907311f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.108141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205765-664f-4874-87e9-1f90e907311f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.108141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205765-664f-4874-87e9-1f90e907311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.130263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7f205765-664f-4874-87e9-1f90e907311f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:34.131505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7f205765-664f-4874-87e9-1f90e907311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:34.167829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:34.222622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 171/336] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:34.390955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6692-1e44-4094-b6c8-9e54853ba3e6 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.391324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6692-1e44-4094-b6c8-9e54853ba3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.391497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6692-1e44-4094-b6c8-9e54853ba3e6 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.393093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 166/337] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:34.398197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf02c5-f58a-4236-87aa-129fc539fde6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.398481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf02c5-f58a-4236-87aa-129fc539fde6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.398752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf02c5-f58a-4236-87aa-129fc539fde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.422579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4bdf02c5-f58a-4236-87aa-129fc539fde6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:34.423658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4bdf02c5-f58a-4236-87aa-129fc539fde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:34.497595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43fbd9-dd1f-474b-bfe5-6e729fc8cb1b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.497881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43fbd9-dd1f-474b-bfe5-6e729fc8cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.498138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43fbd9-dd1f-474b-bfe5-6e729fc8cb1b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.499507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 167/338] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:34.506471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cd2c0-7b4f-4144-b0b7-69e81fbe6674 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.506755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cd2c0-7b4f-4144-b0b7-69e81fbe6674 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.507031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cd2c0-7b4f-4144-b0b7-69e81fbe6674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.508717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 172/339] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:34.538492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8d8cd2c0-7b4f-4144-b0b7-69e81fbe6674 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:34.539725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8d8cd2c0-7b4f-4144-b0b7-69e81fbe6674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:34.600519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:34.624376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 168/340] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:34.775221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986be71-9222-4b14-aa4a-299d61ab216a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.775506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986be71-9222-4b14-aa4a-299d61ab216a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.775780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986be71-9222-4b14-aa4a-299d61ab216a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.777360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 173/341] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:34.783357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.783759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.784288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.809315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986be71-9222-4b14-aa4a-299d61ab216a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:34.809797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:34.811177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:34.922651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 169/342] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:34.976853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:34.995059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986be71-9222-4b14-aa4a-299d61ab216a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.007038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.051374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e467b-6d8e-476a-a2b3-c9ab0b33e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.051952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e467b-6d8e-476a-a2b3-c9ab0b33e132 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.052233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e467b-6d8e-476a-a2b3-c9ab0b33e132 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.052508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e467b-6d8e-476a-a2b3-c9ab0b33e132 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.070425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e467b-6d8e-476a-a2b3-c9ab0b33e132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-4a5e467b-6d8e-476a-a2b3-c9ab0b33e132 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:35.075339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:35.076260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:35.077607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 174/343] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:35.079417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582d273-bfe0-4e5b-ae9b-b0abe88ea3d1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.081915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582d273-bfe0-4e5b-ae9b-b0abe88ea3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.081915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582d273-bfe0-4e5b-ae9b-b0abe88ea3d1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.081915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 175/344] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:35.082454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582d273-bfe0-4e5b-ae9b-b0abe88ea3d1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.138668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a689-330e-4ffd-b12a-d4cfcf41b4c8 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.153165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:35.182749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db03f0-da17-4a1e-b1dc-086dc4d90574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.183399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db03f0-da17-4a1e-b1dc-086dc4d90574 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.183732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db03f0-da17-4a1e-b1dc-086dc4d90574 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.184161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db03f0-da17-4a1e-b1dc-086dc4d90574 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.196977 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db03f0-da17-4a1e-b1dc-086dc4d90574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-67db03f0-da17-4a1e-b1dc-086dc4d90574 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:35.203339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:35.204253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:35.205038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 170/345] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:35.206329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.206517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.206856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.213729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582d273-bfe0-4e5b-ae9b-b0abe88ea3d1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.231668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:35.232859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:35.262962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.263653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.264054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.264325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.276853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:35.283437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:35.284337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:35.285593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 176/346] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:35.286272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.313962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:35.325542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 171/347] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:35.327120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.433450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110221d6-7cf5-4d36-82ff-80b266573655 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.453656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fe6dd-b59c-4103-bb1e-66e02a8d38fa None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.455366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:35.476325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.477106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.477389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.477696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.478367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:35.478671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:35.480325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:35.490733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 177/348] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:35.491654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.496865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.497648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.498079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.498451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.499274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:35.499736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:35.502427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:35.512558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 172/349] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:35.513516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.580657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:35.623865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e917f1f-a8b7-43bf-9308-cbc519daccf9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.644873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b33bf-6d80-48d9-9638-c3dfbc59c611 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.672672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.673309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.673629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.673908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.674707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:35.675014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:35.676539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:35.686797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 178/350] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:35.688042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.688933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.689372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.689840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.692730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.705163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:35.712237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:35.713157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:46:35.714004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:35.716294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 173/351] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:35.717241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.751657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:35.823078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c43c7-92ae-4d6e-a75c-7b728b1a58dc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.855993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6d67-35c6-4b1a-969c-071dd943842f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.863606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:35.869719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.870175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.870790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.871168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.908893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.910074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.910192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.910400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.958669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:35.960153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:35.997170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.001209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.029501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.092808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 179/352] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:36.094520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.115600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 174/353] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:36.116769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.133812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.224458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052d8c-d54c-46bc-a26d-50e5c69a1da9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.243603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5d93b-dcf3-4012-8923-51a7f0116a60 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.264464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.265105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.267414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.267414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.272183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.280997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:36.284496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.285118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.285406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.285709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:36.286068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:36.286948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:36.287740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 180/354] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:36.288390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.374141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.376347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.396661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.406925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229a8b-eb5a-48ef-bc6f-e3bc586e06da None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.447211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.448101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.448385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.448644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.449305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:36.449634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.451012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.462574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 181/355] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:36.463410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.505042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 175/356] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:36.506279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.539161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.593376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f87c14-b5ff-4818-be75-ec4c87432b75 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.633284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298af915-fc88-49ef-93c5-a476fb5f2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.633546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26ed07-f208-4776-97c2-fa3370659234 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.633909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298af915-fc88-49ef-93c5-a476fb5f2272 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.634187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298af915-fc88-49ef-93c5-a476fb5f2272 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.634459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298af915-fc88-49ef-93c5-a476fb5f2272 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.635572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298af915-fc88-49ef-93c5-a476fb5f2272 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.637009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298af915-fc88-49ef-93c5-a476fb5f2272 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.638680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 182/357] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:36.639825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d605cd-fee4-4bc6-bfb4-702cdbda7fd0 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.640101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d605cd-fee4-4bc6-bfb4-702cdbda7fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.640993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d605cd-fee4-4bc6-bfb4-702cdbda7fd0 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.642498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 183/358] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:36.643215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d605cd-fee4-4bc6-bfb4-702cdbda7fd0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.672362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.682002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.682660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.682947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.683339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.684369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.685868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.689285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 176/359] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:36.689285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.767343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d605cd-fee4-4bc6-bfb4-702cdbda7fd0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.803280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.803520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.804120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.804305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.804598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.806047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:36.806047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.807932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.813490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c9109-1ee2-4910-bd75-445ad74d29c3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.817914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 184/360] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:36.817914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.860474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.861220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.861602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.861883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] PATH_INFO: `/v3/users/beb5e5060d73438886c5cd89ae6aee81` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.863036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] RBAC: Authorizing `identity:get_user(user_id=beb5e5060d73438886c5cd89ae6aee81)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.864472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.866111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 177/361] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/users/beb5e5060d73438886c5cd89ae6aee81 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:36.866884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:36.923869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:36.942223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995292e-55ea-48bd-b8c9-498b836bbb1f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:36.981613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb049d2-d559-48f3-bb0d-88729f1b79bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:36.982316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb049d2-d559-48f3-bb0d-88729f1b79bc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.982599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb049d2-d559-48f3-bb0d-88729f1b79bc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.982870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb049d2-d559-48f3-bb0d-88729f1b79bc None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.984045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb049d2-d559-48f3-bb0d-88729f1b79bc None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:36.985702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb049d2-d559-48f3-bb0d-88729f1b79bc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:36.987706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 185/362] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:36.988996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:36.989283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:36.989587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:36.994167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e4ea6-77ed-46c0-99a4-a7077789c507 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.010844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:37.012069 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:37.030603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.031269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.031553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.031834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.037642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.039099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.040679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 178/363] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:36 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.041444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.069712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.078810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 186/364] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:37.079469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.154591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965e400-040b-4799-8fb2-38ccb5a59c3d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.168838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.188411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f838b-4dbd-45bc-a44c-983a4c48c012 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.192712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.193352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.193664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.193941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.195035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.196898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.204239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 179/365] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.204239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.225369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.226065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.226290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.226566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.300972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.302463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.305869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.316840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b1527-3fc2-4e3b-90c2-c437d0c101e5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.356220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.356827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.357106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.357381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.358136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] RBAC: Adding query filter params (role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0, user.id=beb5e5060d73438886c5cd89ae6aee81, scope.system=all) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:37.358446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.360165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.371472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 180/366] 10.222.0.23 () {56 vars in 1308 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/role_assignments?user.id=beb5e5060d73438886c5cd89ae6aee81&scope.system=all&role.id=d20f3e89a1aa4fa58b9d6cedc13ed5f0 => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.372247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.398021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.420447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 187/367] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:37.421698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.495737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490d5dd-209a-4420-a233-8f954a8dd90b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.535368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.536017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.536300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.536466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79817e6-1bb6-4f77-885a-c638d498b0ef None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.536562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.537717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.539644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.545870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 181/368] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.545870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.569835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.579584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.580655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.581644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.581644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.582502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=c63e8d69313f433695ce236bb79eba17) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:37.583219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.584917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.592263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 188/369] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.593156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.649342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.681379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7f6c2-c7b9-469e-9113-da2b0f0aa1bb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.728857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.730318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.730318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.731675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.732654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=c63e8d69313f433695ce236bb79eba17) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:37.732985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.735389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.738413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be611825-cf79-42ab-abb6-62d012b1bcdf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.747966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 182/370] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.749003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.801850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.802506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.802787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.803061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.819040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:37.824716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:37.825748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:37.826549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 189/371] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:37.827206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.889652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:37.920979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b77149-fafe-41fc-94de-3982a5da951d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.967515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:37.968180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:37.968534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:37.968813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:37.970080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:37.971796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:37.973806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 183/372] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:37.975331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:37.990619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae0999-feea-414c-bfe4-637aa447356d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:37.999942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:38.032174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:38.032937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:38.033318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:38.033723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:38.034531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:38.034913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:38.037153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:38.049508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 190/373] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:38.049508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:38.174340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a923dd-3914-41d1-84a6-a80ed9ff3b8e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:38.233089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1ead-22ef-4737-bc62-6ea3bffc69b4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:38.254884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:38.255524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:38.255810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:38.256090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:38.256833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:38.257149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:38.258605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:38.272401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 184/374] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:38.278461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:38.292962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:38.294047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:38.294549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:38.295054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:38.295165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:38.296825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:38.298615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:38.300766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 191/375] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:38 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:38.309081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:38.365889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:38.411278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4993-a98e-4cd9-bbd1-66945236ceb9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:38.475141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:38.475769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:38.476055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:38.476333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:38.478739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a0b9a-71ec-454a-8e54-fa4712f0cc64 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:38.543251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:38.543924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:38.544208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:38.544486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:38.545220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:38.545555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:38.549291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:38.559652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 192/376] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:38 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:38.603367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:38.604869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:38.633928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:38.734563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 185/377] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:38.742004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:38.937659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d260e75-dbf5-42c4-9e07-1206e9d18a87 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:38.939224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:38.988216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:38.988863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:38.989201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:38.989522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] PATH_INFO: `/v3/projects/51e33b7d1c8d453f9d66204642c03a87` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:38.990678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] RBAC: Authorizing `identity:get_project(project_id=51e33b7d1c8d453f9d66204642c03a87)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:38.992113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:38.994106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 193/378] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:38 2022] GET /identity/v3/projects/51e33b7d1c8d453f9d66204642c03a87 => generated 344 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:39.000305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:39.121707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ead5cd-4173-4c20-a356-4e0de076eeeb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:39.135852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:39.158846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81bbcb-674b-4fe6-9084-22f0ae299185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:39.159476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81bbcb-674b-4fe6-9084-22f0ae299185 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:39.159765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81bbcb-674b-4fe6-9084-22f0ae299185 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:39.160047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81bbcb-674b-4fe6-9084-22f0ae299185 None admin] PATH_INFO: `/v3/groups/60952bf16457405c97604a6f3923b647` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:39.165038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81bbcb-674b-4fe6-9084-22f0ae299185 None admin] RBAC: Authorizing `identity:get_group(group_id=60952bf16457405c97604a6f3923b647)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:39.166587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81bbcb-674b-4fe6-9084-22f0ae299185 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:39.173232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 186/379] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:38 2022] GET /identity/v3/groups/60952bf16457405c97604a6f3923b647 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:39.174587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:39.297229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103494d-2b32-41f8-bd5a-30583edad8f8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:39.338555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:39.339224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:39.339517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:39.339799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:39.340533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] RBAC: Adding query filter params (group.id=60952bf16457405c97604a6f3923b647, role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=51e33b7d1c8d453f9d66204642c03a87) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:39.340846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:39.342310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7aa1-7f79-4dbb-b33a-33a49d2cefba None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:39.350148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 194/380] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:39 2022] GET /identity/v3/role_assignments?group.id=60952bf16457405c97604a6f3923b647&scope.project.id=51e33b7d1c8d453f9d66204642c03a87&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:40.204490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3ceae-579a-4253-9fca-77ca555b0134 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:40.205344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3ceae-579a-4253-9fca-77ca555b0134 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:40.205787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3ceae-579a-4253-9fca-77ca555b0134 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:40.207561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 187/381] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:40.218924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ee6f2-ff02-4994-ba10-1fe826e554fe None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:40.219414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ee6f2-ff02-4994-ba10-1fe826e554fe None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:40.219854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ee6f2-ff02-4994-ba10-1fe826e554fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:40.246440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-283ee6f2-ff02-4994-ba10-1fe826e554fe None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:40.247876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-283ee6f2-ff02-4994-ba10-1fe826e554fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:40.281395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:40.338815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 195/382] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:40.691890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8f5b8-bb1e-4451-bf42-48467d0e013e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:40.692289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8f5b8-bb1e-4451-bf42-48467d0e013e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:40.692461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8f5b8-bb1e-4451-bf42-48467d0e013e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:40.694129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 188/383] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:40.698922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2d25-16b5-4854-89c4-8ee2fd10494a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:40.699217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2d25-16b5-4854-89c4-8ee2fd10494a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:40.699500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2d25-16b5-4854-89c4-8ee2fd10494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:40.727795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-086c2d25-16b5-4854-89c4-8ee2fd10494a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:40.729070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-086c2d25-16b5-4854-89c4-8ee2fd10494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:40.754656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8f5b8-bb1e-4451-bf42-48467d0e013e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:40.863339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 196/384] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:40.950554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8f5b8-bb1e-4451-bf42-48467d0e013e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.001246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.001923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.002204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.002480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.005183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e99445-f570-414c-9c6d-2e295fac18ab None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.005704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e99445-f570-414c-9c6d-2e295fac18ab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.006174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e99445-f570-414c-9c6d-2e295fac18ab None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.007947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 197/385] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:46:41.012917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.013421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.013928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.019883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:41.029500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:41.030383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:41.031311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 189/386] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:41.034515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.057275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:41.058504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:41.117885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.176840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 198/387] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:41.177737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.200515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2040b-7e63-4e27-9c85-4aa9fc25009b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.232196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.240477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.241085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.241370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.241676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.242720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:41.243032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:41.244849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:41.255148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 190/388] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:41.261158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.346683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e1fb0-7ba0-4d4b-80bc-bbfc762dc975 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.381754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91bcef-e48c-4d0a-8848-6bd7929dca1d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.385825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.389410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b197207-7ae9-491b-8fce-08909a8a9989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.390228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b197207-7ae9-491b-8fce-08909a8a9989 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.391361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b197207-7ae9-491b-8fce-08909a8a9989 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.391574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b197207-7ae9-491b-8fce-08909a8a9989 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.406803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b197207-7ae9-491b-8fce-08909a8a9989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7b197207-7ae9-491b-8fce-08909a8a9989 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:41.411785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:41.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:41.414147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 199/389] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:41.414801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e6aff-5979-4991-ac2f-d3248a49d5fd None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.414801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e6aff-5979-4991-ac2f-d3248a49d5fd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.415137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e6aff-5979-4991-ac2f-d3248a49d5fd None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.420370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 200/390] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:41.420370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e6aff-5979-4991-ac2f-d3248a49d5fd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.434607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830d7d2-75c9-46bf-bb49-4fd971a75c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.435554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830d7d2-75c9-46bf-bb49-4fd971a75c3b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.436186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830d7d2-75c9-46bf-bb49-4fd971a75c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.436661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830d7d2-75c9-46bf-bb49-4fd971a75c3b None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.438621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830d7d2-75c9-46bf-bb49-4fd971a75c3b None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:41.440678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830d7d2-75c9-46bf-bb49-4fd971a75c3b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:41.443031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 191/391] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:41.444491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.445078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.445583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.464846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:41.466095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:41.518463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.535246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e6aff-5979-4991-ac2f-d3248a49d5fd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.538905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 192/392] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:41.539884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.580988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.581668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.581955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.581955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.583386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:41.583849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:41.585545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:41.597337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 201/393] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:41.598380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.667718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.675116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d9d2-91a4-409a-aa84-1f64857a3cea None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.715985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.716990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.717420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.717875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.719803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:41.721640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:41.723595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 193/394] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:41.724658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.740379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170911d-4732-4d1c-9a00-4737003fb3c2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.790091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.791154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.791445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.791726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.806619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:41.813883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:41.814799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:41.815579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 202/395] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:41.816232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.826216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.852591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ff456-55a2-485e-9842-d1dd729078db None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.893789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.895130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.895642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.896056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.897342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:41.898969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:41.900849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 194/396] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:41.901704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:41.938250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686bb06-c773-4267-a390-9ba2b357e7f1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:41.939001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:41.977250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:41.977942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:41.978218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:41.978496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17/groups/790475329aed42fcb159ad2762a9e3bf/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:41.980383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] RBAC: Authorizing `identity:create_grant(project_id=c63e8d69313f433695ce236bb79eba17, group_id=790475329aed42fcb159ad2762a9e3bf, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:41.981847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:41.995240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 203/397] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Sep 10 20:46:41 2022] PUT /identity/v3/projects/c63e8d69313f433695ce236bb79eba17/groups/790475329aed42fcb159ad2762a9e3bf/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:41.996043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.019464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d7cf-ff33-4e88-a891-d5ffe487d1f0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.060881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.061418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.061732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.063992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.064746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:42.065057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.066519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.076041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 195/398] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:42.076913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.082253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:42.128678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc09bd1-68d1-4c1b-8b0a-0d1419351601 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.172629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e212-2621-4f4c-8a33-706e64baed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.173731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e212-2621-4f4c-8a33-706e64baed23 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.174029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e212-2621-4f4c-8a33-706e64baed23 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.174315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e212-2621-4f4c-8a33-706e64baed23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.183846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:42.190495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838e212-2621-4f4c-8a33-706e64baed23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:42.192577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8542d-82e5-44dc-b85b-378d66ee4c8a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b838e212-2621-4f4c-8a33-706e64baed23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:42.197397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:42.198504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:42.199492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:42.205631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 204/399] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:42.206901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dc9c5-52eb-4a2a-bb3d-c190ae7184d5 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.207182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dc9c5-52eb-4a2a-bb3d-c190ae7184d5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.207449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dc9c5-52eb-4a2a-bb3d-c190ae7184d5 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.208723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 205/400] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:42.209406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dc9c5-52eb-4a2a-bb3d-c190ae7184d5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.233804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.234947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.235396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.235674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.236811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.238310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.239930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 196/401] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:42.240689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.321425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dc9c5-52eb-4a2a-bb3d-c190ae7184d5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.345787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f417-c6de-466b-b88b-ada7e9680f37 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.352294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:42.364256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420ae85-999e-4acd-b335-44201cb81ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.364895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420ae85-999e-4acd-b335-44201cb81ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.365181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420ae85-999e-4acd-b335-44201cb81ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.365450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420ae85-999e-4acd-b335-44201cb81ab2 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.366589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420ae85-999e-4acd-b335-44201cb81ab2 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.368152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420ae85-999e-4acd-b335-44201cb81ab2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.374914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 206/402] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:42.376343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.376449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.376730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.381082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.381719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.381998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.382267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.382915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:42.383225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.384573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.393588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 197/403] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:42.394359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.399163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:42.400361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:42.425135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:42.471216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 207/404] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:42.472378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.522368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8e217-7373-4776-820c-cb30016560dc None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.573820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.575126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.575772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.576600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17/groups/790475329aed42fcb159ad2762a9e3bf/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.579559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de32ea-41a4-46fd-862a-3f069b89e5b7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.580192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] RBAC: Authorizing `identity:create_grant(project_id=c63e8d69313f433695ce236bb79eba17, group_id=790475329aed42fcb159ad2762a9e3bf, role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.582946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.607759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 198/405] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Sep 10 20:46:42 2022] PUT /identity/v3/projects/c63e8d69313f433695ce236bb79eba17/groups/790475329aed42fcb159ad2762a9e3bf/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:42.608621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.616269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:42.638033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.638739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.639073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.639353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.641897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.643991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.647454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 208/406] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:42.654356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.758339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:42.786342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f449b-edb6-424d-b9fc-1c3c87a2abcf None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.834164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2729b-0a4c-462c-abfe-eb8ade983866 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:42.838766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.839416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.839695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.839968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.841079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.842555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.845161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 199/407] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:42.846548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:42.886369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:42.887416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:42.887930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:42.888407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/groups/790475329aed42fcb159ad2762a9e3bf/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:42.891313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] RBAC: Authorizing `identity:create_grant(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c, group_id=790475329aed42fcb159ad2762a9e3bf, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:42.898552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:42.921964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 209/408] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Sep 10 20:46:42 2022] PUT /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/groups/790475329aed42fcb159ad2762a9e3bf/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:42.921964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:43.025797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30c8d-1325-4cba-bc25-db54451452c7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.035469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:43.099960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.100529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.100812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.101089 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.128243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8f54b-2e86-4fff-bc01-4997d0038dca None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.130634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:43.136391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:43.137281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:43.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:43.139357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 200/409] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:43.147482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:43.172169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:43.194646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.195585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.195883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.196170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.197656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:43.199272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:43.205485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 210/410] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:42 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:43.209425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:43.316970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47186-dbe0-4f9a-9b3a-d110ca557830 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.328413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb1cf1-6da3-4855-9e84-94f60e746bd3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.351494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:43.368845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.370278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.370762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.371277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/groups/790475329aed42fcb159ad2762a9e3bf/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.373755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c, group_id=790475329aed42fcb159ad2762a9e3bf, role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:43.376232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:43.381249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.382072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.382406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.382672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.383462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:43.383895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:43.385373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:43.397497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 211/411] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Sep 10 20:46:43 2022] PUT /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c/groups/790475329aed42fcb159ad2762a9e3bf/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:46:43.403404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 201/412] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:43.409700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:43.546628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83634e2-ee13-4d89-bba2-205ddc289b3f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.588370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.589415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.589885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.590306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] PATH_INFO: `/v3/groups/60952bf16457405c97604a6f3923b647` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.591647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] RBAC: Authorizing `identity:get_group(group_id=60952bf16457405c97604a6f3923b647)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:43.593850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:43.596147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 212/413] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:43 2022] GET /identity/v3/groups/60952bf16457405c97604a6f3923b647 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:43.601484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:43.715694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:43.745814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cef250-28e8-4a81-a074-f3d34b8c8b5a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.782701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9bff1-1b9f-4d1b-912c-47b24cc1123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.783341 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9bff1-1b9f-4d1b-912c-47b24cc1123a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.783621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9bff1-1b9f-4d1b-912c-47b24cc1123a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.783893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9bff1-1b9f-4d1b-912c-47b24cc1123a None admin] PATH_INFO: `/v3/projects/51e33b7d1c8d453f9d66204642c03a87` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.785161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9bff1-1b9f-4d1b-912c-47b24cc1123a None admin] RBAC: Authorizing `identity:get_project(project_id=51e33b7d1c8d453f9d66204642c03a87)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:43.786638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9bff1-1b9f-4d1b-912c-47b24cc1123a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:43.788536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 202/414] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:43 2022] GET /identity/v3/projects/51e33b7d1c8d453f9d66204642c03a87 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:43.794186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:43.848881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:43.920605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be343eb-42d9-49f9-8dfa-ab2db8c97b9c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:43.976673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd04af-c7f8-4435-b957-4a32d2695c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:43.977604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd04af-c7f8-4435-b957-4a32d2695c5b None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:43.978147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd04af-c7f8-4435-b957-4a32d2695c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:43.978764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd04af-c7f8-4435-b957-4a32d2695c5b None admin] PATH_INFO: `/v3/projects/51e33b7d1c8d453f9d66204642c03a87/groups/60952bf16457405c97604a6f3923b647/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:43.985740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd04af-c7f8-4435-b957-4a32d2695c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=51e33b7d1c8d453f9d66204642c03a87, group_id=60952bf16457405c97604a6f3923b647, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:43.987374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd04af-c7f8-4435-b957-4a32d2695c5b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:44.009959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 213/415] 10.222.0.23 () {58 vars in 1442 bytes} [Sat Sep 10 20:46:43 2022] PUT /identity/v3/projects/51e33b7d1c8d453f9d66204642c03a87/groups/60952bf16457405c97604a6f3923b647/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:46:44.414410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3bc11-3355-46bc-884f-c727fc2a5d5b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:44.414946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3bc11-3355-46bc-884f-c727fc2a5d5b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:44.414946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3bc11-3355-46bc-884f-c727fc2a5d5b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:44.417367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 203/416] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:44.425749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:44.426249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:44.426333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:44.472920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:44.474246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:44.583377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 214/417] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:44.987483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3bc11-3355-46bc-884f-c727fc2a5d5b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:45.137987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3bc11-3355-46bc-884f-c727fc2a5d5b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:45.190246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe063af-508b-404c-adc9-17bba84e64ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:45.190814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe063af-508b-404c-adc9-17bba84e64ef None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.191096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe063af-508b-404c-adc9-17bba84e64ef None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.191371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe063af-508b-404c-adc9-17bba84e64ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.210641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe063af-508b-404c-adc9-17bba84e64ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-5fe063af-508b-404c-adc9-17bba84e64ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:45.216657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:45.217585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:45.218973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:45.223602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 204/418] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:45.224850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:45.299943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:45.386047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac045d0a-3ac1-4b1a-a4ae-57d3ae8f2c0f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:45.393542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a22ed-b71c-4e20-a36f-8b9463734b24 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.393837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a22ed-b71c-4e20-a36f-8b9463734b24 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.394119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a22ed-b71c-4e20-a36f-8b9463734b24 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.396696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 205/419] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:45.400764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285eaa-0032-419c-b1f8-443f2cf947fc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.401060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285eaa-0032-419c-b1f8-443f2cf947fc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.401651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285eaa-0032-419c-b1f8-443f2cf947fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.412372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:45.436481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-25285eaa-0032-419c-b1f8-443f2cf947fc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:45.437765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-25285eaa-0032-419c-b1f8-443f2cf947fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:45.451385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:45.452002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.452194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.453035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.453808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:45.454118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:45.455513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:45.471632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 215/420] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:45.477578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:45.593926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 206/421] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:45.637767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:45.642485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df967198-40f4-4d2e-b0c6-8f98fabc10ec None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:45.668568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0015-33d2-479e-9c81-4b0559dbf718 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.669211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0015-33d2-479e-9c81-4b0559dbf718 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.669722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0015-33d2-479e-9c81-4b0559dbf718 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.671692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 207/422] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:46:45.676892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.677531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.678003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.697288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:45.697889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:45.697995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:45.698198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:45.698469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:45.699726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:45.814766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 208/423] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:46:45.817230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:45.818987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:45.930749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:45.935584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 216/424] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:45.942814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:45.987950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.049920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:46.115779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453b295-67c5-4198-89d4-19e5eadf36ef None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.135907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69e85-bac4-4918-9845-ffab54f40f05 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.188245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.188965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.189248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.189549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.191694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:46.193135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:46.195357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 209/425] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:45 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:46.200438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.210159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.210803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.211079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.211349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.234559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:46.239457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:46.240340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:46.241407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 217/426] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:46.242537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.307729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:46.340529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9ebc3-a0e5-43d4-9c8d-c12eff924ec8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.354715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a1f75-1f82-4bf7-8392-4e6636ee8e2d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.385346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:46.394660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.395290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.395413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.395610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.395924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.396865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.397357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.397874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.400515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:46.402701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:46.405900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 210/427] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:46.406594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.410461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:46.415664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:46.416632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:46:46.417974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:46.422172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 218/428] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:46.423096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.531921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a5ae-1e0a-4b38-8d77-93d1b7e5fda9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.546066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449479e-b47a-4415-bebb-a50b6c1378ce None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.552488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:46.568215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.568909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.569191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.569496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.570219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=c63e8d69313f433695ce236bb79eba17) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:46.570531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:46.571979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:46.577973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 219/429] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:46.578754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.602624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.603597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.604750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.605209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.606406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:46.607183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:46.609913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965204a-a52e-4f80-947d-afe74f3771bb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:46.622789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 211/430] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:46.624970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dfcf7-1208-4f84-ba6f-614859f0f449 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.626221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dfcf7-1208-4f84-ba6f-614859f0f449 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.626691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dfcf7-1208-4f84-ba6f-614859f0f449 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.628394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 212/431] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:46.629148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dfcf7-1208-4f84-ba6f-614859f0f449 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.711878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccd854-df86-48dc-a02e-fd9e7d78c9d1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.727111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:46.757055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.758083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.758625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.759144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.760178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:46.760789 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:46.762517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef23d6b-7a9d-46af-ba30-ca8aad37ea6f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:46.773452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 220/432] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:46.775113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.775597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.776065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.790196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dfcf7-1208-4f84-ba6f-614859f0f449 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:46.804033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:46.805236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:46.829212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ae5a6-f4b2-402e-b402-8f116372ae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:46.829888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ae5a6-f4b2-402e-b402-8f116372ae79 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:46.830171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ae5a6-f4b2-402e-b402-8f116372ae79 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:46.830445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ae5a6-f4b2-402e-b402-8f116372ae79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:46.852428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:46.888884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 221/433] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:46.891573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:46.918963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ae5a6-f4b2-402e-b402-8f116372ae79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:46.920718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ae5a6-f4b2-402e-b402-8f116372ae79 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.003343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.011266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e9aa2-4763-4972-93fc-4946fbd80d05 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.053935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 213/434] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:47.053935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd74420-c1c8-40d2-bbcf-631d0e1b46f4 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.053935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd74420-c1c8-40d2-bbcf-631d0e1b46f4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.053935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd74420-c1c8-40d2-bbcf-631d0e1b46f4 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.053935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 214/435] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:47.060436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd74420-c1c8-40d2-bbcf-631d0e1b46f4 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.064662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.064662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.064662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.064945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.123900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.149138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.151025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.174820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd74420-c1c8-40d2-bbcf-631d0e1b46f4 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.209316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ea314-3cc2-4e9f-9f58-cb34bebadea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.209976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ea314-3cc2-4e9f-9f58-cb34bebadea6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.210253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ea314-3cc2-4e9f-9f58-cb34bebadea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.210522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ea314-3cc2-4e9f-9f58-cb34bebadea6 None admin] PATH_INFO: `/v3/projects/c63e8d69313f433695ce236bb79eba17` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.211633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ea314-3cc2-4e9f-9f58-cb34bebadea6 None admin] RBAC: Authorizing `identity:get_project(project_id=c63e8d69313f433695ce236bb79eba17)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.213049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ea314-3cc2-4e9f-9f58-cb34bebadea6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.214975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 215/436] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/projects/c63e8d69313f433695ce236bb79eba17 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:47.216074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.216351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.216621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.234299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:47.235477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:47.248977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 222/437] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:47.250722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.263069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.307178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 216/438] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:47.308013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.327129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.351437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fea73-c284-4dbf-9248-f8713e9a8c5d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.387962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.388850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.389127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.389410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.400363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:47.405132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:47.406355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 223/439] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:47.407165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.438306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804df08-bd75-4a4f-b73c-b5a646103b21 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.475353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.476025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.476308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.476584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.477787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.479406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.486879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 217/440] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:47.486879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.513588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.521284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84bdf6-4d75-4449-b7b8-cb7bdf1c1de6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.559938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.560637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.560923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.561197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.563251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.564716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.566382 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 224/441] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:47.567148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.571961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.602477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d99f7-c050-4da5-8635-7ae45e31e697 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.637416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.638077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.638342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.638619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.639276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:47.639583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.641804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.649576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 218/442] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:47.650366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.673659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f2f64-e846-4325-b455-a7308930474c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.708129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.708730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.709008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.709280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.710041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=c63e8d69313f433695ce236bb79eba17) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:47.710348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.712065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.717700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 225/443] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=c63e8d69313f433695ce236bb79eba17&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:47.718629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.748794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.767291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad51e-9137-4a1a-80bd-877efef65b24 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.798170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:47.809690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.810346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.810626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.811143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.812717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:47.814371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:47.821070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 219/444] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:47.821070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.842312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289728-ee0f-4bd1-bf84-acb07e3bb886 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.880834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.881509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.881804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.882078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:47.895039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:47.900356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:47.901225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:47.902046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 226/445] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:47.902694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:47.953399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e094-27ad-419d-a4dd-c746ffec6b0a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:47.990056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:47.990679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:47.990953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:47.991226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.020364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e3ce4-f44d-4059-8263-9a21a320d00e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.022216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.055079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.065827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.066446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.066759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.067039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.067772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=3e2944846ed042f0bf3a6b134cb87695, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:48.068082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.069674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.075860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 227/446] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=3e2944846ed042f0bf3a6b134cb87695 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:48.076462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.076860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.078480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.210031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6255d-0f7d-44d9-87d3-2034fd79da7b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.216980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 220/447] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:48.223202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.249154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.249929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.250289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.250565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.251234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:48.251538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.252931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.261872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 228/448] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:48.268648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.285493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.336163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.343157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e448e44-02c3-4709-85d9-972213f8fe15 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.389124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.389323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.389712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.392438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] PATH_INFO: `/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.393120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] RBAC: Authorizing `identity:get_project(project_id=9cadea7fceae4781a4b59c2bd9a8bd8c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.393250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44f2d1-3010-4a12-9f7e-4f2acea33905 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.396139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.397501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 221/449] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/projects/9cadea7fceae4781a4b59c2bd9a8bd8c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:48.403049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.436222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.437002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.437497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.437780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.511805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.513252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.513353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02931383-92f4-40ee-b9a7-cecd7ffabd95 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.537368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.548830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.549488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.549767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.550041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] PATH_INFO: `/v3/groups/790475329aed42fcb159ad2762a9e3bf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.551146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] RBAC: Authorizing `identity:get_group(group_id=790475329aed42fcb159ad2762a9e3bf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.552708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.554540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 222/450] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/groups/790475329aed42fcb159ad2762a9e3bf => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:48.560429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.567902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.613996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 229/451] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:46:48.619809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.658995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118072f5-adb0-49f3-a9a9-a99053aa6da9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.692271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.692990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.693277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.693595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.694331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] RBAC: Adding query filter params (group.id=790475329aed42fcb159ad2762a9e3bf, role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:48.694643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.696094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.701982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 223/452] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/role_assignments?group.id=790475329aed42fcb159ad2762a9e3bf&scope.project.id=9cadea7fceae4781a4b59c2bd9a8bd8c&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:48.718328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03671e-b9fb-45e7-a356-cd7d173ad75e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.755226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.755891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.756173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.756450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] PATH_INFO: `/v3/projects/51e33b7d1c8d453f9d66204642c03a87` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.757673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] RBAC: Authorizing `identity:get_project(project_id=51e33b7d1c8d453f9d66204642c03a87)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.759168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.760885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 230/453] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/projects/51e33b7d1c8d453f9d66204642c03a87 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:48.767738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.827185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:48.873150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079290e1-5b68-4b1c-83b5-620cd266d3f2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:48.909597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f28e-8a86-4f25-afaa-a7425d513705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:48.910266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f28e-8a86-4f25-afaa-a7425d513705 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:48.910549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f28e-8a86-4f25-afaa-a7425d513705 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:48.910824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f28e-8a86-4f25-afaa-a7425d513705 None admin] PATH_INFO: `/v3/groups/60952bf16457405c97604a6f3923b647` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:48.911932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f28e-8a86-4f25-afaa-a7425d513705 None admin] RBAC: Authorizing `identity:get_group(group_id=60952bf16457405c97604a6f3923b647)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:48.913371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f28e-8a86-4f25-afaa-a7425d513705 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:48.914971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 224/454] 10.222.0.23 () {56 vars in 1180 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/groups/60952bf16457405c97604a6f3923b647 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:48.920991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:48.946355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:49.025775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14977d1-d2d7-49ed-b40d-73d4fd98534a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:49.060666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:49.061275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:49.061594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:49.062521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:49.063752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] RBAC: Adding query filter params (group.id=60952bf16457405c97604a6f3923b647, role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=51e33b7d1c8d453f9d66204642c03a87) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:49.064067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:49.065526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06988df3-c3b1-44cd-855e-59d998f4b8d5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:49.071494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 231/455] 10.222.0.23 () {56 vars in 1376 bytes} [Sat Sep 10 20:46:48 2022] GET /identity/v3/role_assignments?group.id=60952bf16457405c97604a6f3923b647&scope.project.id=51e33b7d1c8d453f9d66204642c03a87&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:51.594124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a03621-2316-4765-b139-2dbc7c470584 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:51.594639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a03621-2316-4765-b139-2dbc7c470584 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:51.594740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a03621-2316-4765-b139-2dbc7c470584 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:51.596293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 225/456] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:51.601417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738196-58aa-4d22-bf5d-030d92573302 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:51.601852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738196-58aa-4d22-bf5d-030d92573302 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:51.602016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738196-58aa-4d22-bf5d-030d92573302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:51.620747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0a738196-58aa-4d22-bf5d-030d92573302 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:51.626023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0a738196-58aa-4d22-bf5d-030d92573302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:51.723734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 232/457] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:51.962632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c36f1-6aad-429b-9309-0f8a18981756 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:51.963265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c36f1-6aad-429b-9309-0f8a18981756 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:51.963743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c36f1-6aad-429b-9309-0f8a18981756 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:51.965586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 226/458] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:51.971088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:51.971375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:51.971829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:51.997672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:51.999068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:52.016018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:52.062312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c36f1-6aad-429b-9309-0f8a18981756 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.085759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 233/459] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:52.235539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c36f1-6aad-429b-9309-0f8a18981756 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:52.269589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:52.281380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:52.282294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.285921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.286203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.301135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:52.302697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f8a61450-e72a-457f-8564-d4d6cbb19267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:52.321519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:52.322451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:52.323290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:52.323290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:52.323290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:52.323290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 227/460] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:52.329571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.392209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e313d-b80b-4442-94cd-39f5f248fc11 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.392553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e313d-b80b-4442-94cd-39f5f248fc11 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.392762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e313d-b80b-4442-94cd-39f5f248fc11 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.394241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 228/461] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:52.395256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e313d-b80b-4442-94cd-39f5f248fc11 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.465750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36cf21-8bf7-4e94-ac29-e0835655599c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:52.514116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:52.515946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.516796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.517491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.518275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:52.518739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:52.520172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e313d-b80b-4442-94cd-39f5f248fc11 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:52.520313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28f1b6-9e64-44b7-8ece-286f74c8bc5f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:52.534636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 234/462] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:52.535838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.536218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.536496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.549244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:52.555215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:52.556689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:52.561813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:52.562543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.562869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.563144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.576268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:52.578713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e0fdc423-c0a3-4253-bcd1-e8e2611e8bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:52.595823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:52.597151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:52.598375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:52.598375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:52.598375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:52.598375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 229/463] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:52.598813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40e94f-6d1b-4d1e-a0e6-1fea6f43738a None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.599090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40e94f-6d1b-4d1e-a0e6-1fea6f43738a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.599357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40e94f-6d1b-4d1e-a0e6-1fea6f43738a None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.600704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 230/464] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:52.602273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40e94f-6d1b-4d1e-a0e6-1fea6f43738a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.631347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:52.633600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 235/465] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:52.635077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.725880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40e94f-6d1b-4d1e-a0e6-1fea6f43738a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:52.737814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753301a0-9ffb-4a77-8051-0cd87f9caff8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:52.764891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:52.766868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.767146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.767512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.768251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:52.769921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:52.778215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:52.779009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.779432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.779845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.780773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:52.781241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:52.783068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78025a3f-0e2c-4a62-9845-e554584a6379 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:52.786497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:52.797812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 236/466] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:52.799522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:52.800008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:52.800508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:52.804037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 231/467] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:52 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:52.805030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.821231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:52.822804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:52.949085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 237/468] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:52.949821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:52.998764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:53.035353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab44139-b4d0-4076-ae7a-c37679b55a64 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.076518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.077136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.077486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.077766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.078415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.079850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.104419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 232/469] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:52 2022] POST /identity/v3/users => generated 248 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:53.123228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d218a-e5d5-46fd-8bb0-377754cbd70d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.157358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:53.169258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.170308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.170876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.171464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.184056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.185943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:53.205027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:53.206055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:53.206914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:53.206914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:53.206914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:53.209492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 238/470] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:53.216324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:53.240378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:53.354618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e9110-beb9-4705-b717-36106c1c7a5d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.369926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936685a6-9fdd-44ea-b6b6-c53c4d227faf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.395653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.396538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.397242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.397559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.398362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:53.398672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.400293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d65c4-b7f9-4afb-9927-bef1ec84aa8f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.410121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.410850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.411205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.411554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.416370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 233/471] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:53.417873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f645c-c145-45e5-ba22-bc1907965e99 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.418357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f645c-c145-45e5-ba22-bc1907965e99 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.418806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f645c-c145-45e5-ba22-bc1907965e99 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.420601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 234/472] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:53.422610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f645c-c145-45e5-ba22-bc1907965e99 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:53.423507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:53.428857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.430782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ca12e241-4238-4af8-b558-abb5882bbbc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:53.447702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:53.448696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:53.449864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:53.449864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:53.449864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:53.449864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 239/473] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:53.450289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.450289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.450512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.469976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:53.470423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:53.495973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:53.549486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 240/474] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:53.549486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:53.560627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f645c-c145-45e5-ba22-bc1907965e99 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.635074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.635680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.635892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.636173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.636832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.638343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.667847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 235/475] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:53 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:46:53.734605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21167c57-4cd8-4cec-a63e-4c18d371868c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.787301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.787973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.788258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.788533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.789189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:53.789513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.790983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.805167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 241/476] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:53.811037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:53.895771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:53.941407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6d84e-95f3-425d-87c7-1823a01e1bc9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:53.976589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:53.977209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:53.977581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:53.977861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:53.978518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:53.980364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:53.981013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:54.011266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 236/477] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:53 2022] POST /identity/v3/users => generated 249 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:54.018692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:54.123451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5238e3-ab2d-4964-842d-bb57963d2741 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:54.167835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:54.168755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:54.169182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:54.169669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:54.170551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:54.172570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:54.181554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.sql.core [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 20:46:54.181554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 20:46:54.181554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [parameters: {'id': '93e975b1af924265b7b4371e03cc2b24', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 20:46:54.181554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Sep 10 20:46:54.200694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return ref.to_dict() Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:46:54.201841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application yield resource Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise self.value Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application session.commit() Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 20:46:54.203668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.flush() Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application compat.raise_( Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 20:46:54.204633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:46:54.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:46:54.206423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [parameters: {'id': '93e975b1af924265b7b4371e03cc2b24', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:54.207410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 20:46:54.208442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:54.211003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 242/478] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Sep 10 20:46:53 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:54.216858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:54.352180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f38c7-ccf2-4c87-953a-7b38030da9d6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:54.366870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:54.395317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9349d68-fbcd-4ecc-8fbc-3fd14b7d9801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:54.396205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9349d68-fbcd-4ecc-8fbc-3fd14b7d9801 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:54.396633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9349d68-fbcd-4ecc-8fbc-3fd14b7d9801 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:54.397109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9349d68-fbcd-4ecc-8fbc-3fd14b7d9801 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:54.408652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9349d68-fbcd-4ecc-8fbc-3fd14b7d9801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b9349d68-fbcd-4ecc-8fbc-3fd14b7d9801 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:54.414359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:54.415430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:46:54.416447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:54.418802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 237/479] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:46:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:54.425305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:54.541885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:54.560390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd019f2-ed30-41a0-8787-935706530a56 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:54.609015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:54.609678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:54.609992 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:54.610260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:54.610893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:54.611195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:54.612578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a85d2-8443-4809-9576-8d0b0e2d180a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:54.625600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 243/480] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:55.662216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57c1a5-bb33-448c-aaae-5e58c647e245 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:55.662721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57c1a5-bb33-448c-aaae-5e58c647e245 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:55.662797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57c1a5-bb33-448c-aaae-5e58c647e245 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:55.664361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 238/481] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:55.669630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ca22c-f309-48ad-adc1-0d8912f8cc35 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:55.670047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ca22c-f309-48ad-adc1-0d8912f8cc35 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:55.670216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ca22c-f309-48ad-adc1-0d8912f8cc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:55.696612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-848ca22c-f309-48ad-adc1-0d8912f8cc35 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:55.697847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-848ca22c-f309-48ad-adc1-0d8912f8cc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:55.789105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 244/482] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:55.868910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa1ce4-73d0-49b4-91d9-e1230863325b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:55.869568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa1ce4-73d0-49b4-91d9-e1230863325b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:55.870040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa1ce4-73d0-49b4-91d9-e1230863325b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:55.871564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 239/483] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:55.877680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:55.878042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:55.878326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:55.905252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:55.906785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:55.995319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 245/484] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:56.256531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa1ce4-73d0-49b4-91d9-e1230863325b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:56.369602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa1ce4-73d0-49b4-91d9-e1230863325b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:56.413577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:56.422445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:56.422445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.422445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.422445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.435396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:56.446329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:56.451554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:56.452735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:56.453677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:56.459821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 240/485] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:56.468582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:56.509243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:56.594890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ca41-43d1-4a41-a689-131b620f3e95 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:56.599342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72273ed4-0345-4933-ae5b-07b06d34637d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:56.637166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:56.637825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.638104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.638377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.638745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a830b-2970-493f-b61e-a6bb6c544152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:56.639039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:56.639338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:56.640759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a830b-2970-493f-b61e-a6bb6c544152 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.641237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a830b-2970-493f-b61e-a6bb6c544152 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.641646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3998cc-9767-4fbd-94b9-9c6099d30b5a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:56.641940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a830b-2970-493f-b61e-a6bb6c544152 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.651658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 241/486] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:46:56.653895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c2aae-289f-465d-bf27-15b6a727969e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.654392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c2aae-289f-465d-bf27-15b6a727969e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.654890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c2aae-289f-465d-bf27-15b6a727969e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.656709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 242/487] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:56.658397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a830b-2970-493f-b61e-a6bb6c544152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:56.661007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c2aae-289f-465d-bf27-15b6a727969e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-750a830b-2970-493f-b61e-a6bb6c544152 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:56.664285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:56.665167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:56.665989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:56.670148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 246/488] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:46:56.672200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.672492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.672763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.691923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:56.693116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:56.758345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:56.767867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 247/489] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:56.768562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:56.802908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:56.814730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c2aae-289f-465d-bf27-15b6a727969e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:56.883838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:56.884490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.884775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.885045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.904485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:56.907783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:56.927404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d15d7-800a-468e-bbf9-88d7aecf77b2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-d89cb8e2-8bc2-44d8-8b28-b1ece5ca24a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:56.929871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:56.930737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:56.931573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:56.931573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:56.931573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:56.931573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 243/490] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:56.937526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94aad1-3af5-46a6-99c3-6dd9fe88370f None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.937526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94aad1-3af5-46a6-99c3-6dd9fe88370f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.937526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94aad1-3af5-46a6-99c3-6dd9fe88370f None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.937526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 244/491] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:56.937526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94aad1-3af5-46a6-99c3-6dd9fe88370f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:56.979362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:56.979978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.980314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.980594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:56.981826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:56.982133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:56.985576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913394e-12ef-4a0a-b7db-631c4168cef0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:56.994551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 248/492] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:56.995903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:56.996196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:56.996478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.021445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:57.022725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:57.069668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94aad1-3af5-46a6-99c3-6dd9fe88370f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.083111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.099932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 249/493] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:57.100961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.118465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.119080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.119357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.119801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.120453 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:57.121136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.122857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.139931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 245/494] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:57.141073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.149536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.226104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca84d92-6ae8-45ab-9965-85618e23e671 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.243583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071742d-82b8-44d0-9c7e-5400f9c9f816 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.265907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.266574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.266856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.267235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.278392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.278640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.279086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.279372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.279642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.279932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.290071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:57.294805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:57.297547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:57.299437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:57.300855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:57.309641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:57.311890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:57.311890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:57.311890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:57.311890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 250/495] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:57.311890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.312531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 246/496] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:57.312531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.351533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.370495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.406105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f44ae7-a330-4943-8ed6-a507a6228387 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.418842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637e617-9e03-4241-b5dc-f8eaa406bf98 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.440770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.441380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.441687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.441955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.442623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:57.443144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.445196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.459513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 247/497] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:57.459693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.459812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.459906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.459906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.459906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.468171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.469864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:46:57.482512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:57.483997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:57.484982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 251/498] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:57.486390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.586596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091a23-1e0c-46ea-b502-143e1f791866 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.591849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.595841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7bf6-2f3a-426d-bad7-d7af07c19fa8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.616849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.626366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.626985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.627267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.627541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.628183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:57.628486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.629887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.631877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.632515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.632795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.633070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.643764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 248/499] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:57.644573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.644757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:46:57.649976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:57.652274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 252/500] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:57.653683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.746546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6965-1169-4973-ad14-be6f53bd6f5d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.751244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2209a75-6f89-41cf-b704-ff5c870e0a23 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.782819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.783448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.783732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.784020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.785442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.786067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.786342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.786612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.787326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:57.787624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.789054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.794115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.795561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7603b858-fea4-4b92-8902-afd5bcf89851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:57.811056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:57.812879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:57.813743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:57.813743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:57.813743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:57.813743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 249/501] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:57.813958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72ef50-a43e-484b-a10f-da637ee59030 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.814424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72ef50-a43e-484b-a10f-da637ee59030 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.814424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72ef50-a43e-484b-a10f-da637ee59030 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.815948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 250/502] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:57.816648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72ef50-a43e-484b-a10f-da637ee59030 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.830314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 253/503] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:57.831155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.849396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.861284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:57.923491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72ef50-a43e-484b-a10f-da637ee59030 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.934133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb036e1-00c9-4ae6-9e74-5a83b23a9dc1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:57.957156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.957812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.958088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.958367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.965505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.966932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.968637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:57.969242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.969558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.969836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:57.970492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:57.970795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:57.972167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-c3da252a-668c-4158-a29e-226e9935a363 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:46:57.979222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:57.980126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:46:57.981002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:57.981534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 254/504] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:57.981603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 251/505] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:57.981667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:57.981745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:57.981974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:57.982254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.000518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:58.001735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:58.067635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 252/506] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:46:58.067837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.068563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.089335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655a0f-a521-4d3f-babc-6d772c9aa4d4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.099276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.127104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.127646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.127929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.128203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.128842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:58.129144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.130569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.143064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 255/507] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:58.144108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.185912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe97444-b6d1-4b82-98ac-e97927b7dae5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.220689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.221303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.221620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.221897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.254885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc9e5-1382-4d0c-b128-5cd4e1c1bd16 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.291861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.292466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.292748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.293024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.293116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.293775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:58.294091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.294586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.295524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.304709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.319530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 256/508] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:58.319530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.331141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.404322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 253/509] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:58.406265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.423544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc787f-676e-490d-bb2b-93186f263a93 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.456987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.457720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.457931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.458204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.468471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.469971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:58.485578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:58.486559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:58.487414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:58.487414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:58.487414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:58.487414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 257/510] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:58.487414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.508240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec3c0e-bd89-42cd-86ed-a2a374f948b6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.536512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.537852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.544464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a3405-560c-413c-be12-490eee90f841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.545405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a3405-560c-413c-be12-490eee90f841 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.545862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a3405-560c-413c-be12-490eee90f841 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.546290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a3405-560c-413c-be12-490eee90f841 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.554693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a3405-560c-413c-be12-490eee90f841 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.556469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a3405-560c-413c-be12-490eee90f841 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-563a3405-560c-413c-be12-490eee90f841 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:46:58.568734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:58.569678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 254/511] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:58.571025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a3405-560c-413c-be12-490eee90f841 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.589626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530614b3-31b8-413c-9ea0-96f81957abae None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.622480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.623395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.623676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.623950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.672030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a3405-560c-413c-be12-490eee90f841 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.693061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.694536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.704843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.705479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.705773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.706055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.716474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.718801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:58.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:58.734337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:58.735198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:58.735198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:58.735198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:58.735198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 255/512] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:58.735818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.744997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.781066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:58.796493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 258/513] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:58.797331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.842002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92225232-40e9-4f52-ac45-d90d44c11dbe None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.888999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.890461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.891168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.891601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.897999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6db95-5b04-494d-ae99-2fa61fa3bdef None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:58.903247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.904705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:58.920984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:58.922383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:58.923243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:58.923243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:58.923243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:58.923243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 256/514] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:58.923243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.937864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63877af-127a-43b5-894a-119015acb047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:58.938482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63877af-127a-43b5-894a-119015acb047 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:58.938763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63877af-127a-43b5-894a-119015acb047 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:58.939042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63877af-127a-43b5-894a-119015acb047 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:58.939682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63877af-127a-43b5-894a-119015acb047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:58.939986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63877af-127a-43b5-894a-119015acb047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:58.941368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63877af-127a-43b5-894a-119015acb047 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:58.953676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 259/515] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:58.954472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63877af-127a-43b5-894a-119015acb047 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:58.966316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.023620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.031702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9255a48-4d88-487c-8e16-733e2d388310 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.054699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63877af-127a-43b5-894a-119015acb047 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.064179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.064790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.065071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.065348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.066103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:59.066417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.068402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.087772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.088380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.088662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.088940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.089605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:59.089920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.091299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.103490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 260/516] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:59.104288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.110932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 257/517] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Sep 10 20:46:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:59.110932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.174720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.207534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515b1a2-206e-4f3c-adec-94eaa649b91f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.213581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f51980-c5fb-4762-84b6-b780f06f4f6c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.244566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.245193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.245483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.245858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.247909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.248524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.248814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.249632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.250356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:59.250661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.252043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.256811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.258279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.263491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 258/518] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:59.264827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:46:59.273977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:59.275095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:46:59.276161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:46:59.276161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:46:59.276161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:59.276161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 261/519] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:59.276161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.283848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.369583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbb2f7-0150-48f6-897d-456102ace14c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.375474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c9b6b-67c7-418b-9358-28e56cce2357 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.398164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.404079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.404714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.404998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.405305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.410719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.411318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.411598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.411880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.412518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.412680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.413991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.414161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:46:59.425201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:59.426097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:46:59.426970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 259/520] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:59.427521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.437808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 262/521] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:46:59 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:46:59.438805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.510228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.549397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce753228-9777-495e-9077-4689ea3c6aa8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.560853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8bf8b-b1aa-4721-881f-6a267ba46c43 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.592164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.593443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.593757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.594410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.600438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.601095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.601374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.601750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.609538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.611016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:46:59.624353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:46:59.625369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:46:59.626295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 263/522] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:46:59.631094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.677141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.682706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.684425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.744611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42630f46-e0d7-480b-88c9-615c834d358b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.761412 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.782357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.782976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.783260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.783599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.784238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:59.784544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.785956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.801551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 260/523] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:46:59.801551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.803904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 264/524] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:59.804695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.911945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:59.916645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4873-7e8d-4e4b-af9a-542e16753ca6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.919495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98104d-2fb7-4fba-8941-2c38408ebffa None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:59.956132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.957651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.957651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.957651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.957896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:59.958200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.959645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.965213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:59.965878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:59.966217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:59.966499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:59.967228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:46:59.967535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:59.969541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:59.979143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 265/525] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:59.979982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:59.995651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 261/526] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:46:59.996520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.027626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.087972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187f2a4-5f9a-41e8-b542-7e318555f709 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.107714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13889913-47a9-445a-8c9c-6adcb383c019 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.122296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.122930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.123207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.123475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.133400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.134876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.146628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.147488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.147920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.148343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:00.149517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:00.150681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:00.151690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:00.151690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:00.151690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:00.151690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 266/527] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:00.151690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.163430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.165274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.166937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:00.179844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:00.180766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:00.181646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 262/528] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:00.182805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.267617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999fc97-bf0f-46f1-911b-f2262b98f958 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.287298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.298080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aeaea-75a5-41ee-b450-b2281a5fa568 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.304831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.305580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.305860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.306130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.306847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=acfbe8c6a73d4b4992faba36c9881012) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:00.307263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.308797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.315126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 267/529] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/role_assignments?user.id=acfbe8c6a73d4b4992faba36c9881012&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:00.326540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.336917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.338590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.338988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.339422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.428779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.429022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.431218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.461589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8c4cb-4434-49c1-9c40-e6fd43f78d03 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.503769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.504517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.504903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.505281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.506182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:00.506613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.508275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.533493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 268/530] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:00.533493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.551677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.564932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 263/531] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:00.566109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.678784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc085cdf-455a-4fbc-9724-c4d2350713e8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.704137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabdc67-3a60-4081-b130-e0ac363dba6f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.728535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.729152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.729435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.730006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.730720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:00.731431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.733165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.736422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.745866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.746506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.748715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.748823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.759195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 269/532] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:00.759195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.768247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.770497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:00.788655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:00.789727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:00.790798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:00.793615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 264/533] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:00.794676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:00.875382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:00.879949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb38cbf-0a45-4b03-acba-58ad3ab0efd5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.929788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:00.930447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:00.930723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:00.930990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:00.945657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:00.947151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:00.951373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca76fc5-5e42-43da-856c-14b19e3b406c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:00.962349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:00.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:00.964719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:00.964719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:00.964719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:00.964719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 270/534] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:00.964719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:01.005595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.007076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.007594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.008072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.009489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=ee9a524cfaa141409ff24c149b698be8) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:01.010483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.012711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.014069 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:01.029677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 265/535] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/role_assignments?user.id=ee9a524cfaa141409ff24c149b698be8&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:01.029677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:01.127522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6369405-26ee-4d95-8b8f-84e65456604e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:01.192009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.192639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.192913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.193230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.193964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:01.194267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.195752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.228737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 271/536] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:01.234898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:01.251743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c6ef8-0315-49f5-9473-ea295dde8f59 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:01.268477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:01.291001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.291769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.291880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.292144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.292892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:01.293199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.294654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.309709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 266/537] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:01.316136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:01.384462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8f49b-08ec-4a33-b83d-04d72d3f351d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:01.403400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:01.426074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.426685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.426963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.427236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.427968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=f07ba8c1958e4e2e82cdd4e490011593) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:01.428272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.429721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.445803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 272/538] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:01 2022] GET /identity/v3/role_assignments?user.id=f07ba8c1958e4e2e82cdd4e490011593&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:01.490743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb86a-720a-4b52-9826-f6b9af07c9db None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:01.528679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.529325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.529641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.529910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.538563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.540007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:01.552863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:01.553814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:01.555228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 267/539] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:01.562807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:01.702527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a31760-4cea-4dbe-9c38-342b1d8af145 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:01.744428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.745052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.745329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.745636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.746335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:01.746654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.748173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f25e45-e327-4436-b2d0-7da294575382 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.761759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 273/540] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:01.770609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:01.808295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:01.823158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0412b8-65d9-4ef4-a12c-c8f7a3a99074 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.823913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0412b8-65d9-4ef4-a12c-c8f7a3a99074 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.824296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0412b8-65d9-4ef4-a12c-c8f7a3a99074 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.826033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 274/541] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:01.838909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.839250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.839453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.858061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:01.860142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:01.918755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:01.932651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593feab3-b705-4014-a584-0a25945c0313 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:01.972969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:01.973632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:01.973864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:01.976019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:01.976758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=934be918dc8c4fa59988e5bb22e56eaa) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:01.977070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:01.978548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:01.994726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 268/542] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:01 2022] GET /identity/v3/role_assignments?user.id=934be918dc8c4fa59988e5bb22e56eaa&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:02.007248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 275/543] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:02.297696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:02.411033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:02.415150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c152-2ed9-4f5f-84b2-2274bf7a6abf None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:02.459765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:02.460396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:02.460672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:02.460926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:02.476162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:02.482374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:02.483249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:02.485769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 269/544] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:02.490379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:02.620562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c998945-e86e-4d55-8187-447b6b316a2d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:02.659224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:02.659834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:02.660110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:02.660383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:02.661106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:02.661408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:02.662782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec8bce-48df-42ac-aa77-f2f106810613 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:02.672307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 276/545] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:02.677884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:02.741620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5679-061e-406b-bc6b-4cfaa2eb81ad None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:02.741620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5679-061e-406b-bc6b-4cfaa2eb81ad None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:02.741620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5679-061e-406b-bc6b-4cfaa2eb81ad None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:02.742357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 277/546] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:02.747693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:02.748229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:02.748704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:02.783390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:02.784961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:02.790855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953e591-4513-4ce9-8ada-335881a12850 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:02.824371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:02.861885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:02.864437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:02.864948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:02.866557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:02.894570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:02.895965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:02.908224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 278/547] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:02.926877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:02.927771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:02.928606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:02.928606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:02.928606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:02.932483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 270/548] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:02.933935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.059148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867774c-7eb3-42fc-bda7-a48bd10432ae None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.104601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:03.105225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:03.105545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:03.105824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:03.106468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:03.106771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:03.108145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:03.128761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 279/549] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:03.134075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.158174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:03.268632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.288456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:03.312786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe907cb2-7da2-4e4a-bdd1-463b7f2bb8eb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.381223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:03.382264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:03.382868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:03.384236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:03.398755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:03.403170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:03.427617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4067-ea27-4126-8eb3-c3951e38f3b6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:03.429489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:03.430565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:03.431487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:03.436033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 271/550] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:03.441105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.507114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:03.507720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:03.508004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:03.508281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:03.521729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:03.538651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:03.543693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:03.544614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:03.545612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:03.553936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 280/551] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:03.561000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.566958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81cb5f-8af0-4373-85cc-c984269ae24c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.604249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:03.605204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:03.605712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:03.606204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:03.607225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:03.607755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:03.610193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:03.636709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 272/552] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Sep 10 20:47:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:03.644171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.685252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:03.737936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2584f48-a476-4174-a587-f438d97ea78a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.803386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cecb1a-b02e-471e-b15f-aba1f7d43d83 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.814340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:03.814998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:03.815290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:03.815578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:03.816422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:03.816742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:03.818213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:03.836153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 281/553] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:03.843587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:03.846934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:03.860872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817667d3-649e-44ad-ad1a-cf1f0b25a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:03.861893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817667d3-649e-44ad-ad1a-cf1f0b25a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:03.862473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817667d3-649e-44ad-ad1a-cf1f0b25a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:03.862943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817667d3-649e-44ad-ad1a-cf1f0b25a2d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:03.991780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7314d-acd5-43a7-9fa8-e23aa8a6d0a5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:03.997110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817667d3-649e-44ad-ad1a-cf1f0b25a2d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:03.998617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817667d3-649e-44ad-ad1a-cf1f0b25a2d4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.037221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.039116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.039618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.040086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.042242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.056704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.059056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f18d15ef-0aeb-43cd-b6a9-f1ba47054ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:04.078445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:04.079425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:04.080269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:04.080269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:04.080269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:04.083508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 282/554] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:04.094326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02632a4-f2c8-477b-870b-045e2e982386 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.094875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02632a4-f2c8-477b-870b-045e2e982386 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.095281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02632a4-f2c8-477b-870b-045e2e982386 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.097196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 283/555] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:04.102331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02632a4-f2c8-477b-870b-045e2e982386 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.126286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 273/556] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:47:04.127883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.128378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.128847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.151645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:04.153967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:04.187353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.242979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 274/557] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:04.244311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.277416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02632a4-f2c8-477b-870b-045e2e982386 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.327743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.329013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.329767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.330707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.331837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:04.332378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.334808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64423c53-94b7-4c60-8e97-6553a24be40d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.353895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 284/558] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:04.355195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7431-af8d-4b35-9827-2bed20e13d2b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.355490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7431-af8d-4b35-9827-2bed20e13d2b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.355999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa7431-af8d-4b35-9827-2bed20e13d2b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.362252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 285/559] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:04.363446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.363736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.364066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.381086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bcc6c-17d5-45ef-847f-7a3e9ca102d5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.386765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:04.387965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:04.423989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.424748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.425027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.425306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.435673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.436937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.438573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:04.455165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:04.456185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:04.457173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:04.457173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:04.457173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:04.457173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 275/560] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:04.457419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.466940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 286/561] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:04.468917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.480857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.613246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd9ba6-95dc-49c4-a38d-cb7ba4333b51 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.631018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5443f6f-4329-4d8a-a0d6-48b82f02127b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.660782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.662023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.662762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.663778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.664769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:04.665394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.667638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.672827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.673517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.673815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.674088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.680747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 287/562] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:04.681417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.688488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.690019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-bed91f29-6383-44ba-99bd-c71475a29f2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:04.703577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:04.704545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:04.705540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 276/563] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:04.706397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df7d32-4689-4dbb-b369-29e4299b5702 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.706565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df7d32-4689-4dbb-b369-29e4299b5702 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.707491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df7d32-4689-4dbb-b369-29e4299b5702 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.709840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 277/564] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:04.709840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df7d32-4689-4dbb-b369-29e4299b5702 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.725052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.785175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.798687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979baf-3ad3-4f5d-a5ab-9a3fdea620a7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.818895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df7d32-4689-4dbb-b369-29e4299b5702 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.832364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.833025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.833305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.833618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.843557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:04.848731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:04.849853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 288/565] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:04.851037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.853671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.854130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.854410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.854680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.861943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.863446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-6edb2a0c-5ef6-4a07-bab0-fe08577ba1bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:04.874440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:04.875339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:04.876206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 278/566] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:04.876719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.876898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.877164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.899549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:04.900746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:04.949836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:04.956832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bbd7b-cf36-4c05-8343-56300a9aa5fd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:04.969597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 279/567] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:04.970432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:04.991150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:04.991756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:04.992026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:04.992571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:04.993070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:04.993379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:04.994838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.013375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 289/568] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.018213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.039991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.085440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f363a91-abde-421e-b56e-b65bb23d1504 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.120670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516990-e614-46f0-b88f-c48f439c6e25 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.122941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.123574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.124457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.125035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.135177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:05.141214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:05.142182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:05.143012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:05.147254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 280/569] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:05.147882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.158469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.159083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.159378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.159632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.160283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:05.160584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.162000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.175134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 290/570] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.175134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.194594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.256183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2dfed-1543-487a-846c-908273b9bdae None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.276990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.280754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68761e0-5bd7-465f-9045-4ebd1d6d580e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.291474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.292082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.292359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.292630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.293313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:05.293643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.295056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.312888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 281/571] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.313723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.316697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.317329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.317637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.317915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.328416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.329902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:05.344165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:05.345102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:05.346060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:05.346060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:05.346060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:05.346060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 291/572] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:05.346060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.412996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.416922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327aa76-ac04-4e97-9ea4-229f2aec3152 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.446534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f2e4-0300-4433-884a-a9d62929ec89 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.451655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.452302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.452581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.452848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.453522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:05.453823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.455183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.462883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 282/573] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.463653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.485254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.485909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.486186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.486457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.496652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.498128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.506506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:05.512342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:05.513350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:05.515396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:05.515396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:05.515396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:05.515396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 292/574] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:05.515396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.567533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3b484-494e-4cb8-9046-dfd011068637 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.602028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.602656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.602934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.603206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.612941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:05.615206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3dc8-abd5-42c4-a2c1-fdccdc7400f7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:05.618122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:05.619083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 283/575] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:05.619955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.638061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.653259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.653895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.654177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.654449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.655722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=233de9dbbb7a48159a1b0acdcd9880b3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:05.656040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.657499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.662775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 293/576] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/role_assignments?user.id=233de9dbbb7a48159a1b0acdcd9880b3&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.662775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.727504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229d860-249d-4ef0-802c-9b6891bd2e32 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.729279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.769850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.770478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.770762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.771038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.771804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:05.772113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.773681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.788568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7251a0-0fff-455a-a856-69b63ec1f561 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.793416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 284/577] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.793416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.830961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.831606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.831883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.832152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.844004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.845490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:05.866812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:05.867802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:05.868662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:05.868662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:05.868662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:05.869600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 294/578] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:05.870419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.913384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c283b8-5f46-4b20-82a4-42d1c10b0411 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:05.913959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:05.955818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:05.956708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:05.957144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:05.957590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:05.958381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:05.958843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:05.960348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:05.976360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 285/579] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:05.977444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:05.992006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd6165-bf77-4897-b5fb-59f3614b5e63 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.004379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.032623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.033313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.033529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.033801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.034896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:06.035199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.037008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.046899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 295/580] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:06.047706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.085913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0146de-d53f-4b50-85aa-0ba539e1453d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.121902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.122535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.122815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.123085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.130518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.131947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:06.143690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:06.144616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 286/581] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:06.145624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.162532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.166454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff49e71-8bf7-4412-b914-b69abc567be6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.203397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.203995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.204277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.204553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.205195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:06.205523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.206892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.219531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 296/582] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:06.220351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.246045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.267021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e0ac4-c41e-4424-ac65-bd942a24fc47 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.303872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.304519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.304798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.305072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.313312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.314827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:06.326079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:06.327555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:47:06.328517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:06.329057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b6405-e5f3-4d70-8ca6-1bd9ad6def07 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.332688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 287/583] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:06.333570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.365105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.365770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.366047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.366313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.376670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.378157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:06.394554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:06.395531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:06.396435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:06.396435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:06.396435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:06.396435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 297/584] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:06.396435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.410533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.444643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b19ac-43b7-4f13-b561-bf63b9e51553 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.480444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.481056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.481336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.481847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.483812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:06.484129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.485616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.498239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 288/585] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:06.499015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.501882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.511880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310293c-560b-4ca7-a830-5482382c3916 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.546779 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.547405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.547686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.547982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.556190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.557883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:06.570553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:06.571765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 298/586] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:06.572769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.622611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3d8f-e245-4663-9dde-c72715f58920 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.649404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.663862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.665171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.665700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.666207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.667299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:06.668095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.669778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.682982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7c1f6-f6ba-4b8f-a8ee-af06166d3617 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.694336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 289/587] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:06.695199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.717148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.718460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.718787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.719063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.719731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:06.719995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.721358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.733019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 299/588] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:06.733858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.762899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.805216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f0c91-53f7-4392-ba33-72522cb46371 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.837413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecf670-a228-4c50-9c8f-8b1f55b80e22 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:06.844076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.844809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.845222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.845541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/acfbe8c6a73d4b4992faba36c9881012/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.860134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=acfbe8c6a73d4b4992faba36c9881012, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.861611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.868570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:06.877159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 290/589] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:06 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/acfbe8c6a73d4b4992faba36c9881012/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:47:06.878018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:06.878485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:06.879179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:06.879542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:06.879889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:06.880723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:06.881103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:06.882682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:06.921502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 300/590] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:06.922866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.023581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becc7ec-1625-45a2-84cb-794c05fb1e95 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.039037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.072630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb93ddd-5d85-485d-a30d-fc44c39225db None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.074157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.074557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.075255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.075947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.088908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.090706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:07.109758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:07.110790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:07.111697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:07.111697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:07.111697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:07.111697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 291/591] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:07.111942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.118281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.119041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.119426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.119798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.130275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.132116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:07.146660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:07.148599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 301/592] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:07.150991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.217966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.235865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb56d-2ba4-4d05-84f0-a98f23f2bd1b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.261553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01777f-f878-4e2d-b086-787c2f8a4d46 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.276373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.276939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.277227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.277526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.291292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.292896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.303083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.303860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.304286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.304663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.305407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:07.305853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.307327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:07.309497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:07.310470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:07.311974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:07.311974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:07.311974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:07.311974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 292/593] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:07.312891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.324759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 302/594] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:07.325934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.330574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.435136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f88a-9f93-4677-9b0d-4c19b706ce5d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.444701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f6f4-ccbb-4fe7-b7a1-ac329c4172ee None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.458750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.476227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.477328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.477834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.478333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.479245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:07.479835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.481654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.490019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.490685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.490965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.491230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.492180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:07.492488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.494843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.507663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 293/595] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:07.508399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 303/596] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:07.509259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.512609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.598686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.634472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7f85f-fc51-46c4-9018-ab4ae18d2346 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.651451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdd084-eb83-481b-a59f-b1766c4856d5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.678086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.678725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.679011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.679284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.689901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.692137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.698409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.699081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.699321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.699594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:07.706403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:07.707289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:07.708163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:07.710955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 294/597] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:07.712271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.712939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.714950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:07.731672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:07.732771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:07.733752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:07.733752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:07.733752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:07.737623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 304/598] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:07.738494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.764118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.836857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1cbd0-aea3-4218-9b09-f8cc203202e7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.864390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca671-4c77-40ed-8950-e33cf8a757dc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:07.873028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:07.883112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.886313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.886608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.886894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.896082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.897952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:07.918090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:07.919054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:07.919975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 295/599] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:07.920458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:07.925722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:07.925722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.925722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.925722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.926114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:07.926114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:07.927176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50e947-f352-4e3f-8602-ffd92ce615fe None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:07.944655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 305/600] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:07.944655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4be8-e0ea-45f0-931e-cd3531f85000 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:07.944655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4be8-e0ea-45f0-931e-cd3531f85000 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:07.944655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4be8-e0ea-45f0-931e-cd3531f85000 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:07.944655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 306/601] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:07.944655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4be8-e0ea-45f0-931e-cd3531f85000 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.033377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:08.037012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a44f8e-154b-4742-bdbf-3518724af141 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.054489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4be8-e0ea-45f0-931e-cd3531f85000 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.073487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.074119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.074399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.074673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.075642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:08.075642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.077852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.090262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.090875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.091164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.091420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.092098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:08.092402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.093614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 296/602] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:08.094662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.094969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4b4ab-2beb-47ed-ab79-9a9bce2c7b46 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.106691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 307/603] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:08.107885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.108232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.108590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.126996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:08.128114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:08.156194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:08.197745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 308/604] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:08.203395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.210838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2093cd-2305-46a6-af41-a160d63512e6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.255674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.256826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.257292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.257765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/ee9a524cfaa141409ff24c149b698be8/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.277284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=ee9a524cfaa141409ff24c149b698be8, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.279730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.287870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:08.305881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 297/605] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:08 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/ee9a524cfaa141409ff24c149b698be8/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:47:08.307928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.341599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a852-9e4e-4c9a-8185-12243ab72837 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.393968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.394582 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.394869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.395148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.402876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.404381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:08.417591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:08.418587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:08.419915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 309/606] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:08.428585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.524286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920bf21-5653-4a10-9566-92a2ae3cacf4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.532758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:08.554567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a2281-c382-43bf-90a3-b2bbce402c42 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.575595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.576284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.576566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.577419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/f07ba8c1958e4e2e82cdd4e490011593/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.589662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:08.597659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.598683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.599421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.600137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.601266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:08.601877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.602380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=f07ba8c1958e4e2e82cdd4e490011593, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.603788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.603972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.618445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 310/607] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:08.619642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.624092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 298/608] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:08 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/f07ba8c1958e4e2e82cdd4e490011593/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:47:08.631310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.728668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09a3fc-acc3-4cb9-8171-44d20f9d4f5d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.750923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ded7-87cd-468b-b537-8caf67bbc462 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.768531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.769417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.769879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.770282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.782671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:08.788417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:08.789648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:08.790485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:08.792804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 311/609] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:08.799796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:08.805263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:08.806544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:08.806544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:08.807002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/934be918dc8c4fa59988e5bb22e56eaa/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:08.842961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=934be918dc8c4fa59988e5bb22e56eaa, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:08.843950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:08.871881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 299/610] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:08 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/934be918dc8c4fa59988e5bb22e56eaa/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:47:08.884924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:08.952879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816811-6aba-416b-a7fe-c6383b27a7d3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:08.999366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:09.000204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.000496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.000773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.001516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:09.001792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:09.003296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fb1d5-8231-4eea-8180-311f7021a557 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:09.016237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 312/611] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:09.021786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:09.024784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:09.139014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699e980-b79f-49ea-ad41-95508638d415 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:09.175699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:09.176347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.176631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.176905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.188175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:09.189655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:09.204768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b76f-0331-4394-affe-ecb17893c319 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.205277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b76f-0331-4394-affe-ecb17893c319 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.205366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b76f-0331-4394-affe-ecb17893c319 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:09.208212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:09.209309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:09.210616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:09.210616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:09.210616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:09.211107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 313/612] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:09.211952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 300/613] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:09.212305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.212603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.212885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.218528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:09.238153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:09.239660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:09.282946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:09.331838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd02225-54ec-4511-a32c-006fa9180e62 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:09.370441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:09.371070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.371352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.371629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.372556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:09.372870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:09.375139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:09.390104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 301/614] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:09.396441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:09.420399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 314/615] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:09.552011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6819-4825-48c0-8406-7b4d16eaedbd None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:09.576673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:09.608988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:09.609664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.609948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.610223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.623299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:09.625542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:09.642522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:09.643670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:09.644784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 302/616] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:09.653082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:09.732755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:09.799706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786381d-59c0-41fd-ba45-120d9e9d90e2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:09.839189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:09.841361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:09.842216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:09.843030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:09.843958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:09.844705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:09.845017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:09.847483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:09.868862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 315/617] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Sep 10 20:47:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:09.875511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.001531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:10.005152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76bed4-8473-4145-a102-8206c64ec962 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.047735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.048801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.049182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.049601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.061569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:10.063462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:10.081613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:10.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:10.083816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:10.083816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:10.083816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:10.085394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 316/618] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:10.090563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.143195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d3fd-5060-47f3-bac2-9a980c5a52bb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.212630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dadfc7-2f88-4b06-8676-7f659186a8ad None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.250745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.251513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.251897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.252275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.253016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:10.253423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:10.254167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.254825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.255114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.255306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:10.255558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.270173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 317/619] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:10.276077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.284391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:10.303682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:10.304670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:10.305550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 303/620] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:10.306252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:10.310938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.380103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:10.492160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbca4f-1d94-4f7e-a898-85fc1cf19563 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.531979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.532819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.533202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.533609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.535625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ddfdf-c93f-4389-9aef-7cb346fee39c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.541760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:10.543274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:10.556392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:10.557330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:10.558248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:10.558699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 318/621] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:10.565956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.622774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.623404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.623707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.623989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.624665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:10.624972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:10.626391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:10.651727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 304/622] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:10.662451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.665812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:10.699561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821ea6c-2b2c-4c5b-ab83-fa71cdb57ccd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.737658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.738403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.738739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.739221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.739919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:10.740230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:10.741683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:10.756798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 319/623] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:10.762290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:10.874837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e7462-2338-4832-9b60-16d12e41fae9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.910575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:10.918334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665787-c331-40b4-8d23-410005555157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:10.919286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665787-c331-40b4-8d23-410005555157 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:10.919775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665787-c331-40b4-8d23-410005555157 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:10.921564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665787-c331-40b4-8d23-410005555157 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/233de9dbbb7a48159a1b0acdcd9880b3/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:10.945734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:10.950077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73665787-c331-40b4-8d23-410005555157 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=233de9dbbb7a48159a1b0acdcd9880b3, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:10.951618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73665787-c331-40b4-8d23-410005555157 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:10.955820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6923-34c6-4d78-b2f0-c086b9d5bb3b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:10.987046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 320/624] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:10 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/233de9dbbb7a48159a1b0acdcd9880b3/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:47:11.020809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:11.021493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.021872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.022156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.042545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:11.044067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b7e15f39-6cde-4b91-ae9d-6c654c93cd5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:11.060077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:11.061091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:11.062185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:11.062185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:11.062185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:11.066068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 305/625] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:11.072093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665787-c331-40b4-8d23-410005555157 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:11.115869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2581-47ec-413e-9015-75d6f2132000 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.116185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2581-47ec-413e-9015-75d6f2132000 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.116435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2581-47ec-413e-9015-75d6f2132000 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.118111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 306/626] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:11.129602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf71d-aa89-40b2-8088-2045cd0bdd3b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.131488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf71d-aa89-40b2-8088-2045cd0bdd3b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.131817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcf71d-aa89-40b2-8088-2045cd0bdd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.182798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bcfcf71d-aa89-40b2-8088-2045cd0bdd3b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:11.183990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bcfcf71d-aa89-40b2-8088-2045cd0bdd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:11.230175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665787-c331-40b4-8d23-410005555157 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:11.270611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:11.271230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.271514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.271790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.272562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:11.272965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:11.274384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:11.287909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 321/627] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:11.299510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:11.325806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 307/628] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:11.327363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223640fa-9f36-452a-a052-4d713f237cfe None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.327655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223640fa-9f36-452a-a052-4d713f237cfe None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.328117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223640fa-9f36-452a-a052-4d713f237cfe None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.333582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 308/629] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:11.336488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.336782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.337067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.363255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:11.368426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:11.369720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:11.469848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2cbc-bcb4-46b0-8235-2b85589f15c8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:11.491503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 309/630] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:11.516122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:11.516755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.517029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.517350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.525512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:11.526960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0b76c670-af50-4a99-bfbf-a4cee7b357d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:11.538378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:11.539336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:11.540297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 322/631] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:47:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:11.547528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:11.559666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:11.628966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a1f4-97b4-4319-8d5d-b3c0364e021e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.629280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a1f4-97b4-4319-8d5d-b3c0364e021e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.629545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a1f4-97b4-4319-8d5d-b3c0364e021e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.631119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 323/632] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:11.641449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.641899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.642313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.680187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded68a0-5fe8-43fd-ae52-9232623a69f6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:11.688562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:11.689772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:11.727752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:11.728295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.728571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.728845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.729710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:11.730011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:11.731452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:11.753350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 310/633] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:47:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:11.754233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:11.791682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 324/634] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:11.792362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:11.837491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:11.852184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:11.904398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b84293-6ad9-46ff-8fca-895d2b97195f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:11.922647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e34c37-ebcd-4963-8c36-becacad041d0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:11.945619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:11.946350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.946704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.947088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:11.958396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:11.963537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:11.964641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:11.965710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:11.967229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 311/635] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:11.968451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:11.968841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:11.969445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:11.969885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:11.970296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.064691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.066674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.084886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f3f74-7470-477b-93ab-7eda372b83fb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.103225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.115813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.126400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.127035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.127314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.127575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.137615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:12.143323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:12.144266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 312/636] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:12.145161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.183949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 325/637] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:12.185557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.253360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eaa0c-faad-4287-a4d1-2ead7714e2bd None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.285415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91b4bc-070b-419d-ab3c-952f617e4d56 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.290826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.291435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.291712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.291985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.292640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:12.292940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.294731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.309011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 313/638] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:12.309011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.316614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.322911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.323836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.324118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.324393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.335783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:12.340527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:12.341526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:12.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:12.347349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 326/639] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:12.348144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.365383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.424802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567041c1-71ee-4746-9321-c7cb9be54d8c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.454538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55db507-bbb2-44e7-a091-e3e26129aedb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.463056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.463666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.464303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.464584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.465601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:12.465916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.467598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.481219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 314/640] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:12.482134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.491280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.491922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.492204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.492479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.503920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.505394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:12.521071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:12.522039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:12.528141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:12.528141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:12.528141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:12.528141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 327/641] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:12.528141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.572854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.598611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb01e-a577-418e-9a0b-957bf153b236 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.610372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.631075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e921a4-a583-4b63-a243-fc6fc8d4cbfa None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.637477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.638413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.638880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.639402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.651656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.653484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.664209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.664811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.665087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.665356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.666031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:12.666351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.667678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:12.672045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:12.673027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:12.673974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:12.673974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:12.673974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:12.679483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 315/642] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:12.679483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.679688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 328/643] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:12.679688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.787447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308394a-6cf7-41b0-a14a-75b84496a93c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.787677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca263b-f025-46b6-b95b-e8d18fb058eb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.795896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.824217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.824778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.825056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.825330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.827069 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.827680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.827958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.828237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.828870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:12.829171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.830566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.836572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.838049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:12.847271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 329/644] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:12.847271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:12.856002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:12.856971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:12.858246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:12.858246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:12.858246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:12.861605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 316/645] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:12.862655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:12.882409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:12.958058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0289e-dcae-40ed-82d4-3256901488ae None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.978808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9c57-c952-499d-a998-c97b940192e5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:12.992959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:12.993608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:12.993890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:12.994168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:12.994806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:12.995108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:12.996503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.013319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 330/646] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.013319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.021114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.022157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.022502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.023109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.035483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.037071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.039054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:13.053349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:13.054379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:13.055262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:13.055262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:13.055262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:13.055262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 317/647] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:13.055262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.122828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ac46c-611a-4b6c-957c-038ba624f431 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.128541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.165610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.166246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.166530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.166836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3511a3-5c0f-482c-b6f1-65e4c51b21e4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.166938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.174947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.176333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:13.188830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:13.189777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:13.190672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 331/648] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:13.191726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.209984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.210600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.210883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.211155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.211796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:13.212100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.213476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.227757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 318/649] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.229495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.287056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.304880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb08661-5b96-4a3e-8cd7-1b0f81653fe3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.342482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.343123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.343300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e087-a1bf-4b09-b860-9d5bb17bfa87 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.343481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.343813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.351548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.352983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:13.364946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:13.365928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:13.366871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 332/650] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:13.367469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.374762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.386219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.387101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.387387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.387662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.388849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:13.389415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.391168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.402811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 319/651] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.403663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.473845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13027-f858-4ee9-bbe9-f482b9b25e50 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.504051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d133e-1d27-46c2-8193-6ba32e51ce4c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.506449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.507059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.507339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.507609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.508346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:13.508654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.510120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.523123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 333/652] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.524328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.533292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.538523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.539161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.539443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.539724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.547118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.548556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-8410ff2e-c455-4df2-9b80-52893eff21a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:13.561055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:13.563068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:13.564060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 320/653] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:13.565312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69856a72-2a41-4414-a4f7-b5672603d1f0 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.565678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69856a72-2a41-4414-a4f7-b5672603d1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.565962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69856a72-2a41-4414-a4f7-b5672603d1f0 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.567793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 321/654] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:13.568714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69856a72-2a41-4414-a4f7-b5672603d1f0 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.615966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.625714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb6b44-af39-41f5-b02e-b32622679c87 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.659791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.660395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.660675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.660944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.661696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:13.662001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.663424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.671515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69856a72-2a41-4414-a4f7-b5672603d1f0 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.681080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 334/655] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.681914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.710137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.710771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.711053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.711334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.720065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.721650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:13.733335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:13.734411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 322/656] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:13.735932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.743990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.784091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c21e5-6fb0-451b-a778-b4797bbe1d19 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.818302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.818901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.819175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.819447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.820487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=acfbe8c6a73d4b4992faba36c9881012) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:13.820797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.822239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0a2d-6140-4e79-9346-f58ae4459d63 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.828476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 335/657] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/role_assignments?user.id=acfbe8c6a73d4b4992faba36c9881012&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.829715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.829999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.830270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.840146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aca34-e03c-4180-acaf-0b0690a2e885 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:13.848395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:13.857984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:13.860756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:13.880870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:13.881780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:13.882222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:13.882666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:13.883570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:13.884231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:13.885923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:13.907741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 323/658] 10.222.0.23 () {56 vars in 1137 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:13.908461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:13.950467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 336/659] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:13.951167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.024447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.048627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f690-1b9a-41f5-b5e5-5e5267ee7e6b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.071945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2ae08-d484-43aa-91d0-8413a5182588 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.095562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.096183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.096467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.096745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.116798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.117411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.117737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.118010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.118742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:14.119048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:14.120488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:14.136917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.142301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 337/660] 10.222.0.23 () {56 vars in 1139 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:14.142301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.181355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:14.182855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:14.252019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e4988-3fe6-4ead-811c-9fce3d3388dc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.269591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.290768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.292135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.292643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.293133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.300395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 324/661] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:14.301528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.371331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.371536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:14.372951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:14.413580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040494c-55d4-414f-b61d-5c19d5561c6b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.457043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.457763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.458047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.458329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.490763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 338/662] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:14.491936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.510239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.537068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:14.538672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:14.599175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253715-6832-4f7a-af7d-408112900172 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.605795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.637369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.638028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.638305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.638578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.652808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 325/663] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:14.652808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.654179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:14.659167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:14.660078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:14.661012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 339/664] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:14.661728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.767838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.778197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdbd72-4f96-47a5-8381-ac1dff41778b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.791168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41d552-aee8-4f9d-b746-dd438fb56649 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.816159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.816791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.817074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.817349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.827867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:14.828358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:14.828546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:14.828776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:14.829057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:14.829849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:14.839390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:14.840829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:14.844866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:14.846039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:14.846972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:14.846972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:14.846972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:14.850580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 326/665] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:14.851414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:14.855982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:14.856906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:14.857819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:14.857819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:14.857819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:14.857819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 340/666] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:14.859467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:14.870848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:14.968155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7089c5-04e0-44d4-96d5-e7b80bb68d2e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:14.987673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04af4f-84d1-4139-9710-6aee5fa3d4c3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.003272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.003907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.004198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.004471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.014864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.016609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.016733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.025301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.025949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.026233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.026511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.027298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.027609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.028971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:15.032637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:15.033585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:15.034484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:15.034484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:15.034484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:15.034484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 327/667] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:15.035104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.042971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 341/668] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.043848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.110338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.146931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453789da-2ce0-409f-a015-d23a81a3c718 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.159485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f900-2aae-4445-96cc-1df2da56f70e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.185800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.186619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.187162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.187162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.187821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.188120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.189507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.195109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.195720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.195999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.196269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.196906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.197206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.198591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.201795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 328/669] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.202887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.218144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 342/670] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.219021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.272329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.319101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f49dc-beb4-460a-8ba1-c217fceed408 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.327759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa66a3-5f9f-41e9-8d8e-29ec7061a6a7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.346924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.355760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.356480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.356771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.357121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.358462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.358778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.360160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.369265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.370169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.370652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.371122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.371787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 329/671] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.372589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.386495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.388396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:15.405197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:15.406175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:15.407074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:15.407074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:15.407074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:15.408777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 343/672] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:15.409966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.484820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58347c-2248-4f6e-b42e-117021a5f904 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.512517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced49eac-b7c2-4c87-afae-1c1e9769c97c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.514070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.520665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.521304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.521617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.522011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.528525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.530063 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:15.541857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:15.542767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 330/673] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:15.543682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.556419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.557424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.557813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.558092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.565275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.566770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:15.579466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:15.580359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 344/674] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:15.581300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.600326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.654352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1baf-704c-4c3d-87ce-dacc2119d459 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.690815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.691458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.691741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.692014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.697306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059057e-e8be-4818-acd6-69f1673935c5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.698754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.700210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:15.713601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:15.714568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 331/675] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:15.715464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.739006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.739916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.740416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.740778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.741524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.741904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.743294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.760353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 345/676] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.760353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.791842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.840292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630f339-b1da-4a85-94db-5be88032dc69 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.853941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:15.885749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.886500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.886850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.887218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.888109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.888491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.890861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.908857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 332/677] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.909735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:15.937553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3adeb-a30a-4b87-906e-d77bb05cd629 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:15.973746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:15.974442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:15.974769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:15.975054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:15.975739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:15.976046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:15.977610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:15.992478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 346/678] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:15.993476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.037863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5fc2b-2c29-4705-bd48-e2246c873626 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.077913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.078670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.079013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.079365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.080161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:16.080550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.082421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.118497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 333/679] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:16.118497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.119348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:16.152151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:16.152968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00220a39-8e8c-4ac3-bedd-712d194d53d9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.204142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.204780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.205060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.205332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.212927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.214400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:16.226022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:16.227072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 347/680] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:16.228014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.238028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcdffb-b29a-4ceb-bea3-9d60103dc799 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.273302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.273940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.274223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.274499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.275224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=ee9a524cfaa141409ff24c149b698be8) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:16.275537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.276953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.282775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 334/681] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:16 2022] GET /identity/v3/role_assignments?user.id=ee9a524cfaa141409ff24c149b698be8&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:16.283558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.336703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8410e4-a59f-47cb-8c56-35543c49f5f8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.377913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.378790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.379555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.379555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.380079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=f07ba8c1958e4e2e82cdd4e490011593) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:16.380462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.382029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0106b72-2aab-4ea1-b221-62f1cfed7cee None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.401508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 348/682] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:16 2022] GET /identity/v3/role_assignments?user.id=f07ba8c1958e4e2e82cdd4e490011593&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:16.401508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b506af-09b2-48cb-8788-b2533bebc746 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.401508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b506af-09b2-48cb-8788-b2533bebc746 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.401508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b506af-09b2-48cb-8788-b2533bebc746 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.401508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 349/683] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:16.401508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b506af-09b2-48cb-8788-b2533bebc746 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.414047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:16.427956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:16.442930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6b5f4-3bc6-461d-acdd-29a6da075eb8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.487722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.488369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.488629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.488911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.489697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=934be918dc8c4fa59988e5bb22e56eaa) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:16.490010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.491480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f585b41-29a3-4c9f-91fe-b119be86ef5f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.499016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 335/684] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:16 2022] GET /identity/v3/role_assignments?user.id=934be918dc8c4fa59988e5bb22e56eaa&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:16.500515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.500854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.501758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.528440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:16.530305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:16.563255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b506af-09b2-48cb-8788-b2533bebc746 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.609417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.610124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.610553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.610838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.611971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:16.612109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.613681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.640402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 336/685] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:16.641128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 350/686] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Sep 10 20:47:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:16.650434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.747639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:16.765081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8717e19-a92c-45b2-be33-87bd12caeb78 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:16.801428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:16.802079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:16.802363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:16.802637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:16.874504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:16.876034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:16.941543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:16.984429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:16.998345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 337/687] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:17.006684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.078439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:17.152622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb76c-7e30-45a2-a1da-457738732132 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.185931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f89d89-d15a-40a5-b57c-99e38298f656 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.206347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.207082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.207435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.207836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.226289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:17.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:17.233594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:17.234621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:17.236597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 351/688] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:17.241094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.258198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.258940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.259292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.259637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.281805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:17.283483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:17.301313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:17.302503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:17.303571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:17.303571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:17.303571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:17.303785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 338/689] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:17.312620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.366338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79798916-0d36-492e-87dd-2f094014ad96 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.392258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:17.397837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:17.403697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.405388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.406084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.406472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.407335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:17.407761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:17.409252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:17.418652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 352/690] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:17.425360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.517067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53451f9-c81b-4083-853f-d009144a10b6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.558217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.560140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.560576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.561064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.562741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:17.563200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:17.564983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:17.585386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 339/691] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:17.592660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.601015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb013bc-b37f-4619-9e79-e31684e790db None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.666632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.667589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.668024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.668432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.694244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:17.695965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:17.718769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:17.719818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:17.720700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:17.720700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:17.720700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:17.729801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 353/692] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:17.730665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.738340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:17.743167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8358aa9-94cd-4e6f-89d7-b5c53d5a128f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.782853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.783521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.783802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.784078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.791601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:17.793073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:17.805862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:17.806837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:17.807713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:17.808181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 340/693] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:17.814514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:17.868993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:17.931512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c963b-4414-463a-928f-dddb7debac97 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:17.975986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:17.976603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:17.976886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:17.977161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:17.977887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:17.978193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:17.979616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:17.992495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 341/694] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:17.999065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:18.003956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cd46d-fb89-45f6-b93f-0036431b8024 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:18.053841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:18.059571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:18.060190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:18.060466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:18.060737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:18.062101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:18.062288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:18.063668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:18.089283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 354/695] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:18.095783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:18.197332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345aa9b6-e226-4b12-b354-2003f337775b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:18.231351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:18.231959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:18.232244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:18.232520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:18.233297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=233de9dbbb7a48159a1b0acdcd9880b3) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:18.233646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:18.235135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:18.241283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 342/696] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:17 2022] GET /identity/v3/role_assignments?user.id=233de9dbbb7a48159a1b0acdcd9880b3&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:18.251436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:18.260266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044bcf7-0cec-4b3d-8ea5-4698926db615 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:18.304359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:18.305002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:18.305327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:18.305555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:18.315216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:18.316777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:18.329529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:18.330572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:18.331552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 355/697] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:47:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:18.336789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:18.478806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:18.501339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bf4c7-e522-414c-833d-6087cbe722a5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:18.546303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:18.547023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:18.547248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:18.547469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:18.548212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:18.548518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:18.550937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:18.573352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 343/698] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:47:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:18.582560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:18.703870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424eae0-4990-48d4-815b-4aa3678f7328 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:18.742637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d60b-8a23-4351-bb21-3db6d7c1af3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:18.743604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d60b-8a23-4351-bb21-3db6d7c1af3a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:18.744147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d60b-8a23-4351-bb21-3db6d7c1af3a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:18.744632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d60b-8a23-4351-bb21-3db6d7c1af3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:18.776472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:18.824780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d60b-8a23-4351-bb21-3db6d7c1af3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:18.827067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d60b-8a23-4351-bb21-3db6d7c1af3a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:18.944670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 356/699] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:18.950642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:19.103033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fa489-3746-401a-a9a7-c08dc3f60a6a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:19.109013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa4a25-718d-4d09-b73c-31b7b733f8be None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.109359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa4a25-718d-4d09-b73c-31b7b733f8be None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.109644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa4a25-718d-4d09-b73c-31b7b733f8be None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.112605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 357/700] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:19.117270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b15e-40ec-405d-9bf9-31c8641f2591 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.117604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b15e-40ec-405d-9bf9-31c8641f2591 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.117877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b15e-40ec-405d-9bf9-31c8641f2591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.139746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-70e3b15e-40ec-405d-9bf9-31c8641f2591 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:19.141238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-70e3b15e-40ec-405d-9bf9-31c8641f2591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:19.194832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1725463-1059-4219-a20f-95f022470314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:19.195492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1725463-1059-4219-a20f-95f022470314 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.195775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1725463-1059-4219-a20f-95f022470314 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.196051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1725463-1059-4219-a20f-95f022470314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.210811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:19.220011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:19.227521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1725463-1059-4219-a20f-95f022470314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:19.228988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1725463-1059-4219-a20f-95f022470314 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:19.233593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 358/701] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:19.260489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db818-68e0-4b52-a516-729865e5ba9b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f1725463-1059-4219-a20f-95f022470314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:19.260700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:19.261640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:19.262598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:19.262598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:19.262598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:19.262776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db818-68e0-4b52-a516-729865e5ba9b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.262776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db818-68e0-4b52-a516-729865e5ba9b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.262902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 344/702] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:19.262969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 359/703] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:19.269443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2bb06-566f-49c4-975b-98f1b6c8a6cf None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.269769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2bb06-566f-49c4-975b-98f1b6c8a6cf None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.270052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2bb06-566f-49c4-975b-98f1b6c8a6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.273569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db818-68e0-4b52-a516-729865e5ba9b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:19.303459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e8a2bb06-566f-49c4-975b-98f1b6c8a6cf None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:19.304996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e8a2bb06-566f-49c4-975b-98f1b6c8a6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:19.381033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 345/704] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:19.382747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cebaa-8300-4127-a655-2a3da9a76e34 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.383244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cebaa-8300-4127-a655-2a3da9a76e34 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.383718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cebaa-8300-4127-a655-2a3da9a76e34 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.389415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 346/705] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:19.391003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.391512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.392000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.414406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:19.416736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:19.450487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db818-68e0-4b52-a516-729865e5ba9b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:19.510125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 347/706] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:19.522559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:19.524188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:19.524823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.525109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.525384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.526081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:19.526387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:19.527761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:19.547794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 360/707] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:19.552676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:19.615545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:19.664227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:19.713999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a14f2-08c2-4bae-b3ce-4bdbd4fde99d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:19.762381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346ff1b-318b-4e30-98b7-89fe43bb18ac None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:19.783434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:19.784202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.784487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.784766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.797407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:19.798900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:19.803341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:19.803979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.804262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.804606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.811353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:19.812796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:19.814711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:19.815635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:19.816535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:19.816535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:19.816535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:19.816535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 348/708] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:19.817580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:19.830091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:19.831187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 361/709] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:19.832084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:19.938690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98929a2a-264b-4903-8477-7b64f3e64f0a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:19.946802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979e898-9413-4508-a3ba-0d463def92e0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:19.948106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:19.949437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:19.977352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:19.978022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.978349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.978617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] PATH_INFO: `/v3/services/network` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.979152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:19.980632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:19.986881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:19.987521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:19.987802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:19.988075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:19.988657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:19.990106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:19.992468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:19.993502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:19.994356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 362/710] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:19 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:19.995181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:20.008811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:20.009908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:20.011476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 349/711] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Sep 10 20:47:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:20.012281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.114637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92f7f-c45e-41a4-bdbf-a86e83c4ecf0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.115354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375eaf7-7a0f-4a8a-87f2-c3e1e0865f71 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.150118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.150729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.151007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.151280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.151957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.152218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.153666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.153872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.154479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.155348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.155732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.156537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.156848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.158270 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.170662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 363/712] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:20.170662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.179120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 350/713] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:20.179120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.198600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:20.214904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:20.269839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712075-6ecf-486f-8c72-19e673e34193 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.288830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824652c7-8848-496a-973a-59ea503a6bda None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.305886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.306785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.307275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.307726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.308574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] RBAC: Adding query filter params (name=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.309150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.310707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.316187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 364/714] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:20.316944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.327808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.328425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.328703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.328976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.329631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.329934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.331293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.335894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 351/715] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Sep 10 20:47:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:20.336668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.427369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:20.434155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a9a7e-8f7e-40c0-9086-f4f048fb9d33 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.443060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:20.452782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d39bc-581e-4768-a95b-cdb092ac51d9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.471202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.471815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.472097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.472373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.473024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.474490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.491202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.491875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.492226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.492502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.493355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=acfbe8c6a73d4b4992faba36c9881012) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.493694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.495296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.499475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 365/716] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:47:20 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:20.500510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.512193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 352/717] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:20 2022] GET /identity/v3/role_assignments?user.id=acfbe8c6a73d4b4992faba36c9881012&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:20.514689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:20.644174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c966d0-48ac-499d-b463-dded87096022 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.650022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4752-6f59-47b7-b3d1-b2f368a9389e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:20.684636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.685284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.685602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.685976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.686828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.687145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.688709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4c4d5-089d-41a3-a334-f984657d108e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.695233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 353/718] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Sep 10 20:47:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:20.708872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:20.710144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:20.710570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:20.711012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:20.711814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:20.712264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:20.713854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cfdc0-1905-4a4c-8b50-5ac223ab76a1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:20.728215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 366/719] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:21.137793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed397fa-f8b1-4dc7-b5a7-57861634c896 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:21.138175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed397fa-f8b1-4dc7-b5a7-57861634c896 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:21.138367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed397fa-f8b1-4dc7-b5a7-57861634c896 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:21.141588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 354/720] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:21.147335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:21.147621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:21.147896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:21.174504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:21.175718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:21.195533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:21.277803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 367/721] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:21.575596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed397fa-f8b1-4dc7-b5a7-57861634c896 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:21.604101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:21.687650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed397fa-f8b1-4dc7-b5a7-57861634c896 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:21.734406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:21.735059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:21.735345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:21.735691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:21.754642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:21.760127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:21.761050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:21.761913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 355/722] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:21.768068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:21.877303 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98795bd9-ca90-4f70-af0a-d88f94c90635 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:21.929305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:21.929947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:21.930225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:21.930496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:21.931155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:21.931454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:21.932815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:21.947743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 368/723] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:21.953539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:22.067256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:22.093823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4d804-d8fe-4376-9595-537545e25167 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:22.139820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:22.141104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:22.141634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:22.142114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:22.160191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:22.162025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:22.184311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:22.185338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:22.186663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:22.186663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:22.186663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:22.188616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 356/724] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:22.198550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:22.215847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:22.350730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6f65b-a668-429c-baf1-2ad8d1b066f6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:22.392185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:22.392717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:22.392996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:22.393268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:22.393939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:22.394239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:22.395609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:22.412607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 369/725] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:22.418529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:22.570029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1d364-e4b8-4a2c-9585-ce6b001366f2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:22.618015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:22.623459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:22.624220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:22.624500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:22.624773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:22.636709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:22.638173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:22.655762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:22.656772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:22.657800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 357/726] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:22.662856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:22.758575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:22.804273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f52e4-50e7-4fb7-bda8-43c07ba49de2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:22.853096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:22.854187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:22.854695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:22.855198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:22.856136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:22.856661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:22.858877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:22.882651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 370/727] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Sep 10 20:47:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:22.887832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:23.046099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b7e2d-7235-41be-a960-7149f389680c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:23.084872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3304-ee1a-40ee-9b06-28c876540209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:23.085504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3304-ee1a-40ee-9b06-28c876540209 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.085793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3304-ee1a-40ee-9b06-28c876540209 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.086065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3304-ee1a-40ee-9b06-28c876540209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.162420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:23.176209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3304-ee1a-40ee-9b06-28c876540209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:23.177700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3304-ee1a-40ee-9b06-28c876540209 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:23.317624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 358/728] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:47:23.322838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:23.375549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6fe6b-6bd5-4fcd-a56d-73d76ec026e5 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.375882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6fe6b-6bd5-4fcd-a56d-73d76ec026e5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.376115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6fe6b-6bd5-4fcd-a56d-73d76ec026e5 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.381892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 359/729] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:23.383333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b79d62-c7ca-4a34-96da-2a0015324dad None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.383625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b79d62-c7ca-4a34-96da-2a0015324dad None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.383903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b79d62-c7ca-4a34-96da-2a0015324dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.411052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-02b79d62-c7ca-4a34-96da-2a0015324dad None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:23.411859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-02b79d62-c7ca-4a34-96da-2a0015324dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:23.496136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee6cec-b17a-4cc1-a754-6e0187ca1656 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:23.532593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 360/730] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:23.538294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:23.547767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad238720-6d96-4d01-bc54-d960b190fac6 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.548062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad238720-6d96-4d01-bc54-d960b190fac6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.548324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad238720-6d96-4d01-bc54-d960b190fac6 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.548800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:23.549442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.549812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.549984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 361/731] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:23.550077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.555285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de41a65-2b79-4aa6-a22c-c6575ff2f462 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.555570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de41a65-2b79-4aa6-a22c-c6575ff2f462 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.555851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de41a65-2b79-4aa6-a22c-c6575ff2f462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.563391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:23.565002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:23.576044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5de41a65-2b79-4aa6-a22c-c6575ff2f462 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:23.577528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5de41a65-2b79-4aa6-a22c-c6575ff2f462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:23.581332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:23.582331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:23.583225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:23.583225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:23.583225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:23.583225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 371/732] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:23.588243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:23.599732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:23.660058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 362/733] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:47:23.661359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9fc37-9c9f-4c9e-9ac5-2b35e9906a5b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.661678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9fc37-9c9f-4c9e-9ac5-2b35e9906a5b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.661958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9fc37-9c9f-4c9e-9ac5-2b35e9906a5b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.669729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 363/734] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:23.687172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf96638-0642-4388-8845-2c515caacb2c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.687454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf96638-0642-4388-8845-2c515caacb2c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.687735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf96638-0642-4388-8845-2c515caacb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.715972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0cf96638-0642-4388-8845-2c515caacb2c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:23.717221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0cf96638-0642-4388-8845-2c515caacb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:23.735127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02caf01e-8c35-43f6-817d-3ef82f6a452d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:23.775576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:23.776565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.777138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.777738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.778585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:23.779079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:23.781126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:23.788225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 364/735] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:23.796118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 372/736] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:23.800543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390acd9f-118f-46bb-88db-1c62dfa8b293 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.800543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390acd9f-118f-46bb-88db-1c62dfa8b293 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.800543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390acd9f-118f-46bb-88db-1c62dfa8b293 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.800543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 365/737] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:23 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:23.802027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:23.804787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:23.805077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:23.805357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:23.835345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:23.836487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:23.854265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:23.912920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 366/738] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:47:23.913934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:23.929128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd620d-8dbe-4dc3-ab8d-bb19a45417f1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:23.930851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:23.999657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.002547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.004054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.004656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.012108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.014711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:24.027413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:24.028297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:24.029217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:24.035615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 373/739] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:24.036680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.060469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657d5d-857c-4fe7-9294-ba697ce77b6e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.103901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.104465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.104749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.105037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.118197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:24.124399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:24.125320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:24.126249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 367/740] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:24.126932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.145207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:24.149994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af661c51-df0a-4415-a9c2-02577f789c30 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.187695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.188396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.188682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.188957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.189647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.191018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.201009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:24.209511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 374/741] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:24 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:24.209511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.242073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c6118-cb4f-41dd-91d6-5074dddab676 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.298307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.299230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.299649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.300076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.300957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:24.301413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.303086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.317361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 368/742] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:24.318499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.346277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d41249-ad5f-4da9-a44a-32c89555b6f9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.383824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.384367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.384649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.384922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.385613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.387348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.405827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 375/743] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:24 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:24.406833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.447547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:24.455743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa936006-ad00-405b-93bd-48e289daa93e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.499366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:24.519457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.521078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.521577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.522298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.524762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:24.525835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.527740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.542292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 369/744] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:24.543570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.555351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30811994-81e3-41f8-a588-545f9bdc1fc2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.597157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.597923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.598213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.598489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.610816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:24.616277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:24.617211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:24.618333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 376/745] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:24.619209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.683228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbb06a-47ed-4b75-bea2-9f482e6bbc56 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.711993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:24.735623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.736234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.736512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.736783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.737538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=233de9dbbb7a48159a1b0acdcd9880b3) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:24.737849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.739274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.745229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 370/746] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/role_assignments?user.id=233de9dbbb7a48159a1b0acdcd9880b3&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:24.749494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.751667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bc97f-deab-4e86-836b-b9a9e6561829 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:24.826509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:24.827656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:24.827934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:24.828200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:24.842647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:24.870699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:24.872253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:24.905499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:24.906501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:24.907386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:24.907386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:24.907386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:24.907386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 377/747] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:24.912672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:24.914943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6128a-0d72-4495-bea5-7043c7b92071 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.019672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.020245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.020523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.020796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.021496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:25.021795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.023216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.039756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 371/748] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:25.047606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.073638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4579c6-8e1a-4c22-a105-56bf20102cf3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.118153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:25.128411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.129321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.129769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.130183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.131230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:25.131706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.134602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.156995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 378/749] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:25.162176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.265448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9455d-c29f-4b9e-9858-c8db16b8d3f8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.288302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:25.329694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.330393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.330672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.330942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.337422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f5ddc-01a0-44ab-bb14-f78670d19cb8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.354835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.356284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.377603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:25.377944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:25.378892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:25.379795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:25.379795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:25.379795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:25.379795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 372/750] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:25.381126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.381639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.382112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.385375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.398517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.400314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:25.423429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:25.424492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:25.425439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 379/751] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:25.434417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.511299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4e93c-f40d-43c2-9296-c87297a856ec None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.538874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:25.549932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.550958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.551461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.551943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.552843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:25.553365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.570310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 373/752] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:25.578764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.617081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae85a3-a0d5-4136-93f1-242a7415b9d0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.628194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:25.668587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.669171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.669619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.671219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.672305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:25.674110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.676436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.706455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 380/753] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:25.712326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.773398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b45889-fb62-4192-ad4d-16e88e6a8fbf None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.841156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.842236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.842715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.843183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.854202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.856734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:25.877145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:25.878140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:25.879060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:25.885725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 374/754] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:25.887538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:25.895608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93543961-3b47-42d5-9465-a4aa338bd768 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:25.923579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:25.972344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:25.972988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:25.973263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:25.973557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:25.993564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:25.995017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:26.015193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:26.016466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:26.017356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:26.017356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:26.017356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:26.017356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 381/755] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:26.022178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:26.073751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:26.098388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c66919-8f89-4a01-adc9-c76d360a12dd None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:26.164064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:26.164682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.164959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.165230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.165984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:26.166287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:26.167709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.197402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 375/756] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Sep 10 20:47:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:26.198903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:26.206743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf44d-af17-4b52-9ad3-724e7b7f6a60 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:26.307197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039480f-8df0-46e6-b593-3d03b27eefed None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:26.310765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:26.311400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.311683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.311956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.312602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:26.312903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:26.314300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.344229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 382/757] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:26.348864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:26.349601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.350003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.350348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.351941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:26.365918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:26.388776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:26.482403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:26.484740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.499037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0309d3-6e25-407a-bc21-d085233dec6b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:26.550754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:26.551407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.551682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.551948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.559265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:26.560743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:26.593694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:26.594672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:26.595559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 383/758] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:26.602633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:26.618690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 376/759] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:47:26.630402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:26.713218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:26.759689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:26.786337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba5d5c-b334-46ef-a69e-056326ec49c3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:26.792670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849922b6-9801-4d8c-916d-786f60c28d11 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:26.830998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:26.831616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.831899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.832015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:26.832224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.832653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.832867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:26.832969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.833177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:26.833279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.834669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf81fb6-d010-41c2-a959-b446b2ab6c8a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.847614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:26.849432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:26.850566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 384/760] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:26.854450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073345e-22e4-44f6-a93b-61704a8446c1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.854664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073345e-22e4-44f6-a93b-61704a8446c1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.854941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073345e-22e4-44f6-a93b-61704a8446c1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.856269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 385/761] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:26.856960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073345e-22e4-44f6-a93b-61704a8446c1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-cdc51e90-9ba8-409c-8917-c59aff6a531e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:26.875915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:26.876923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:26.877875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:26.877875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:26.877875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:26.877875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 377/762] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:26.878619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.878906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.879186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.903486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:26.905307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:26.992758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 378/763] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:26.993810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.006086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073345e-22e4-44f6-a93b-61704a8446c1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.021856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.047122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5fc8c-5e65-4285-be89-ed1d4b66884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.047787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5fc8c-5e65-4285-be89-ed1d4b66884e None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.048062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5fc8c-5e65-4285-be89-ed1d4b66884e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.048364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5fc8c-5e65-4285-be89-ed1d4b66884e None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/acfbe8c6a73d4b4992faba36c9881012/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.050493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5fc8c-5e65-4285-be89-ed1d4b66884e None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=acfbe8c6a73d4b4992faba36c9881012, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.051997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5fc8c-5e65-4285-be89-ed1d4b66884e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.063517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.069299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 386/764] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:26 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/acfbe8c6a73d4b4992faba36c9881012/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:47:27.071405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5927-6f5c-4a31-9ddb-8e06fcc19080 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.072836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5927-6f5c-4a31-9ddb-8e06fcc19080 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.073109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de5927-6f5c-4a31-9ddb-8e06fcc19080 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.077520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 387/765] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:27.081406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.081817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.082196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.109291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:27.111010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:27.138172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b5cf-6b4d-4008-9743-e4f1e26a7080 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.186594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.187859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.188600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.189292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.190594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:27.191341 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.194085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.210831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 379/766] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:27.217960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 388/767] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:27.218279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.286734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.342742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.380133 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a589d-b2b9-4700-b8ec-297016f36ec2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.422376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.423025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.423304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.423579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.431891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.433363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:27.446420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:27.447326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:27.448510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 380/768] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:27.453860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95167bc-2558-486a-8822-d87fb14f2bdc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.458831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.498940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.506570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.507466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.507900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.508352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] PATH_INFO: `/v3/services/network` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.509158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.510709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-4babe4a2-9b67-4fba-97b5-2a0b9ca2159d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:27.520978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:27.522131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:27.523049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:27.528451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 389/769] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:27.530710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c815fb-6ada-4e96-b880-fa169c22ec91 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.531222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c815fb-6ada-4e96-b880-fa169c22ec91 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.531644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c815fb-6ada-4e96-b880-fa169c22ec91 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.533454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 390/770] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:27.536470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c815fb-6ada-4e96-b880-fa169c22ec91 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.570657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b6fb7-1067-423a-b908-40deeed5dba1 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.607587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.608318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.608602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.608879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.609603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:27.609912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.611457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dd92e-c198-47fc-abb6-48c221cd052e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.625066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 381/771] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:27.626304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.626593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.626867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.635364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.646789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:27.647978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:27.650949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c815fb-6ada-4e96-b880-fa169c22ec91 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.688611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.689230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.689552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.689835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.690475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:27.690790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.692582 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.702148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 391/772] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:27.702148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.725039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 382/773] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:27.732220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.764475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.810456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178e81-6c92-4ce1-85c6-9c69cffe90e6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.849598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.850851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.851209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.851556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.852487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.854652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.864866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d96e-9230-4cb8-b70e-b6fd8642c6df None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:27.866889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:27.868076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:27.869098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:27.871527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 392/774] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:27.872397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:27.916706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:27.917330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:27.917875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:27.918158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.918943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=802cad51148347a189b9150bcf4fcf18) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:27.919251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:27.920987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:27.923220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:27.933490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 383/775] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/role_assignments?user.id=802cad51148347a189b9150bcf4fcf18&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:27.933490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.027121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3b47e-5b8b-40ca-9319-ffc4abf56fd0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.063598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:28.077926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.079255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.080148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.080651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.081535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] RBAC: Adding query filter params (type=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:28.082073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.084633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.092921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 393/776] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.094170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.104593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beb39c-2fdc-4121-ad65-50ffdf8c7287 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.148739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.150732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.151121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.151496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.152244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:28.153134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.155466 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.161026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 384/777] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Sep 10 20:47:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.161984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.219912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7200c0-cb49-4448-aeb9-b6d354c63736 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.227523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:28.259939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.260807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.261226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.261687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.270186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4932c8a-7482-4a90-a453-d2c837d92f7e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.273756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:28.278733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:28.279882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:28.280977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:28.283384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 394/778] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:28.284537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.308465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.309500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.309721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.310098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.311051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] RBAC: Adding query filter params (interface=public, service_id=7618f14050ce4791a6e533a0f4a87d2c, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:28.311481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.313039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.325530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 385/779] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/endpoints?service_id=7618f14050ce4791a6e533a0f4a87d2c&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.325530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.345555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:28.409085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3ecf-972e-407c-babd-abd30921fffb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.446830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.447695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.448107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.448521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.449305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:28.449772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.452053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.457776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 395/780] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.458592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c5e8-7484-45ac-800b-f25d88d1f44b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.459037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.489798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:28.501396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.502195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.502574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.502978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.503862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:28.504269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.505783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.517233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 386/781] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.518227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.571781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dc5f5-525d-4201-a3d6-e40088eabe14 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.584892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:28.611820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.612845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.613273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.613762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.614499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.616004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.629651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 396/782] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.631082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.646970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d799d3-8101-4b23-8cea-cbb998788629 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.714023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.714913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.715338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.715775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.716651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] RBAC: Adding query filter params (interface=public, service_id=fadb7e32cbf9432ea4ef143d0e77afe0, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:28.719734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.721438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.733722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 387/783] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/endpoints?service_id=fadb7e32cbf9432ea4ef143d0e77afe0&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:28.741694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.854916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062fac5-e773-4f6d-8659-fb8bdc678f14 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.862583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:28.909407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.910113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.910393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.910665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.926112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:28.928108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:28.941106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac6d7-5f87-4d96-93b3-8f987f26ff08 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:28.948215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:28.949131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:28.950053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:28.950053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:28.950053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:28.950053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 397/784] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:28.958717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:28.995094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:28.995642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:28.995926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:28.996199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:28.996809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:29.001517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:29.006243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:29.008742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 388/785] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/services => generated 772 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:29.128061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031b471-b77b-4acb-ae4f-a932ddbf604c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:29.180965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:29.181912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:29.182338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:29.182778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:29.183588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:29.184040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:29.186157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:29.205445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 398/786] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:29.211161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:29.447253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8a4a7-bc0f-4610-bf8f-c3e49bbefc83 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:29.467764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:29.486196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:29.486904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:29.487191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:29.487480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:29.494823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:29.496257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-80a69d3e-4045-4727-9758-fb90d64c41db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:29.508566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:29.509557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:29.510496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 389/787] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:29.516910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:29.554715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b161d4a-1053-4435-8e2f-421047aee5e6 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:29.555036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b161d4a-1053-4435-8e2f-421047aee5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:29.555270 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b161d4a-1053-4435-8e2f-421047aee5e6 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:29.556610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 390/788] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:29.562263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:29.562537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:29.562803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:29.591623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:29.593064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:29.653202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:29.682548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a7bb-9664-4541-93aa-a52e2c19a99f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:29.706308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 391/789] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:29.849676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:29.850297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:29.850616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:29.850898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:29.851630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:29.851935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:29.853363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:29.872915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 399/790] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Sep 10 20:47:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:29.879379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:29.986163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53d805-c03e-4f69-8853-ec27a7994b74 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:29.994639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:30.022816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:30.023465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.023747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.024024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.034941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:30.036464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:30.051703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:30.052653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:30.054000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:30.054000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:30.054000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:30.054000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 392/791] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:30.058799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.171269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.178063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97126-136f-4bd5-9ba9-b34942207b64 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:30.223810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:30.224440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.224724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.224999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.227594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:30.227916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:30.230942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:30.263039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 400/792] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:30.269518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.273660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:30.349496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba225-1cfc-46c2-a2b1-21483f893cd6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:30.394250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:30.396062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.396352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.396632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.416975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:30.423488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:30.424408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:30.425873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 393/793] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:30.434934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.475978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:30.488146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0f685-9b3b-4b84-9ba0-844fd352e7e6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:30.533061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:30.533889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.534281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.534621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.542414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:30.543857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0e032283-d587-40db-a638-d70e9ae0005a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:30.564761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:30.565774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:30.566663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:30.574268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 401/794] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:30.574990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423f7bf-0ceb-4a0a-9caf-9748d0b5c6d4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.575209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423f7bf-0ceb-4a0a-9caf-9748d0b5c6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.575484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423f7bf-0ceb-4a0a-9caf-9748d0b5c6d4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.578326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 402/795] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:30.579041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423f7bf-0ceb-4a0a-9caf-9748d0b5c6d4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.602322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb26cc-b233-4c89-a5b4-99b6fb7ab66b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:30.643943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:30.644561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.644843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.645246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.645865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:30.646223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:30.647874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696792d-f43f-46e8-8038-33321bf45b40 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:30.663244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 394/796] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:30.663244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.663244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.663244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.674087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:30.685409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:30.687433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:30.723255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423f7bf-0ceb-4a0a-9caf-9748d0b5c6d4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:30.766562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:30.767557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:30.768045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:30.768510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:30.769448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:30.769969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:30.771596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:30.773613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 395/797] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:30.774530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.787131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 403/798] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:30.793656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:30.798929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:30.933488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac07cf-c7c1-4d99-b78a-c1045ce5aa3d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:30.989053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fddbba-b75c-4bd9-8faa-135ace833e89 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.008531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.009498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.009960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.010428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.034837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.036575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:31.042566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:31.067784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:31.068797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:31.069753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:31.069753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:31.069753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:31.076740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.076740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.076740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.076740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/233de9dbbb7a48159a1b0acdcd9880b3/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.077783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=233de9dbbb7a48159a1b0acdcd9880b3, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.078043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 396/799] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:31.078543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:31.079596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.119899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 404/800] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:30 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/233de9dbbb7a48159a1b0acdcd9880b3/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:47:31.154035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:31.196028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.207865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25dfb-e768-4d8c-88bc-4abf74ccf311 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.251214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.251857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.252138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.252410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.253076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:31.253386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.254817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:31.287676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 397/801] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:31.293964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.419250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f2d65-584e-4bab-ba4b-60e1545bcf54 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.436488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880f82-7974-4740-af2d-9fe4ab54f478 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.444885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:31.457932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.458695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.458987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.459271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.468222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.470253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e6b7868c-f906-4fed-84c6-ca292014f147 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:31.483572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:31.484670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:31.485930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 398/802] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:31.487119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ec836-fef4-46c1-8af9-2b0c6c75c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.487119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ec836-fef4-46c1-8af9-2b0c6c75c2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.487119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ec836-fef4-46c1-8af9-2b0c6c75c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.487119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ec836-fef4-46c1-8af9-2b0c6c75c2c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.490656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409cdff-489f-4876-ab20-ce87fb3975c9 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.490942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409cdff-489f-4876-ab20-ce87fb3975c9 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.491223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409cdff-489f-4876-ab20-ce87fb3975c9 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.492763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 399/803] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:31.494844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409cdff-489f-4876-ab20-ce87fb3975c9 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.502414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ec836-fef4-46c1-8af9-2b0c6c75c2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ff0ec836-fef4-46c1-8af9-2b0c6c75c2c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:31.507660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:31.508561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 405/804] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:31.510504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.512245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.512245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.534094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:31.535431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:31.589701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:31.617785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409cdff-489f-4876-ab20-ce87fb3975c9 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.627297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 406/805] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:31.628425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.660500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.661345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.661656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.661974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.662864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:31.663221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.665273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:31.688306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 400/806] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:31.694436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.765274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5df34-a4f6-4ef7-b689-f7d51266de1c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.768725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:31.811408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.812262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.812614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.812949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.814060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:31.814435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.816132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d6bcb-d3f6-4c4f-a3ec-552467d33270 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:31.829501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 407/807] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:31.829501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f04681-8848-49c7-978b-2fefcb09a6d4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.829501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f04681-8848-49c7-978b-2fefcb09a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.829501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f04681-8848-49c7-978b-2fefcb09a6d4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.832684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 408/808] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:31.833945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f04681-8848-49c7-978b-2fefcb09a6d4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:31.839347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae40923-0f28-4902-96c8-dfc9cfae67da None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.877399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:31.878657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:31.884933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:31.885657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:31.886258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:31.960279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f04681-8848-49c7-978b-2fefcb09a6d4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:31.969706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:31.971528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.002742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.003438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.003724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.004140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.014778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.016611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.016860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7a805fa6-6331-48f2-9725-e083719f0050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:32.033040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:32.034071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:32.034971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:32.034971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:32.034971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:32.034971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 409/809] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:32.035722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.035722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.035973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.054231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:32.055418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:32.087021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 401/810] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:47:32.087881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.139276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 410/811] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:32.139816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.161514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.216185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb659e13-4ec2-4bad-9790-7e5b36649df4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.252186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647805f-38ae-49f3-9382-0a14048a3f98 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.262330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.263852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.264974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.266024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.266811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.268602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:32.280350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:32.281369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:47:32.282224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.285268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 402/812] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:32.286321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.290161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.293448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.295133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.295133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.295133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.295773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:32.296211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.298077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.315771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 411/813] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:32.316812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.401407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.408611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed96fa-062d-4482-b69a-e494a7c752e7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.425482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f785b1-5d8a-4f0c-91d7-f7194e9d2b22 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.446530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.447386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.447664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.447943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.459893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.461176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.461445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.461820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.462104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.462382 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.463012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] RBAC: Adding query filter params (name=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:32.463325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.464705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.469872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 412/814] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:32.474978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:32.477235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:32.478604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:32.480360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:32.480360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:32.480360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.480360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 403/815] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:32.480360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.543507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.585398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ccc0-6c84-464a-98b3-8c835d47f16d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.595456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9ddda-03e6-4595-8453-0be8fe076c90 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.620883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.621556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.621841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.622191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.629864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.631291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.632638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.633279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.633591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.634240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.634449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.635792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:32.643913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:32.644825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:32.650711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 413/816] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:32.651311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:32.652260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:47:32.653124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:32.655474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.655614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 404/817] 10.222.0.23 () {56 vars in 1129 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:32.655614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.655780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.756629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68386329-103a-4aaa-a5d9-838ebef1c396 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.762643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91728057-022c-48bd-b960-70d46c0d699d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.777019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.791942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.792881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.793171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.793449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.794120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:32.794449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.796500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.801756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.802583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.803053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.803547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.804412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] RBAC: Adding query filter params (type=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:32.804893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.806463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.808095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 405/818] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:32.808891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.816357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 414/819] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:32.817324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.898223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:32.917203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e6e3d-7c34-426a-b37d-fcdb2f9cf83c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.936819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15af1-0cfe-41f0-877a-82b9bfe21935 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:32.954330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.954949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.955231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.955508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.956332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:32.956674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.958142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.973965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:32.974569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:32.974852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:32.975117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:32.975759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:32.976060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:32.976778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 406/820] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:32.977419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:32.981928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:32.986522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 415/821] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:32.987351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.025989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.091085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad29fdc-5925-405a-9b38-222096a39b00 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.092644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbaaaf-c30c-4894-9b1c-8ad980fdfdba None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.121218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.126424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.127067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.127349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.127635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.128443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.129052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.129425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.129678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.130326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.131677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.134981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.136448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:33.149315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:33.150462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 407/822] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:33.151526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.161500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 416/823] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:47:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:33.162924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.271484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.276842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698c2d8-f471-4a5a-8ac7-52c9222395ee None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.314457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a08f-ff34-4c6a-a154-ab8729d57aff None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.322462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.323392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.323825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.324313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.338259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.339790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.356568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.358110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.358585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:33.358831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:33.359733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:33.360610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:33.360610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:33.360610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:33.360610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 408/824] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:33.361244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.361828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.362734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:33.363235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.365101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.370623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 417/825] 10.222.0.23 () {56 vars in 1124 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:33.371651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.427378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.487654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a4919-b84d-45f7-a9a4-5a021ce8ec8c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.509878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db66c24-a24e-4ea8-ae4c-4e9c3f41673e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.526108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.527091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.527591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.528095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.528974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:33.530074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.531782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.544762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 409/826] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:33.545420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.551272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.551972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.552322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.552682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.553399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:33.553803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.555179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.558149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.567857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 418/827] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:33.568674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.645381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31d606-495c-4745-8e88-352165caf722 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.674163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9fc4d-bb41-40eb-bc59-5d6b63a85979 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.677827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3dff-8f0d-4918-968b-21836e162d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.678434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3dff-8f0d-4918-968b-21836e162d0b None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.678711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3dff-8f0d-4918-968b-21836e162d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.678979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3dff-8f0d-4918-968b-21836e162d0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.679634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3dff-8f0d-4918-968b-21836e162d0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.680982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3dff-8f0d-4918-968b-21836e162d0b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.681422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.701655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 410/828] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:47:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:33.702849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a07e0-b57f-420d-b4d5-8d68da77cd16 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.703137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a07e0-b57f-420d-b4d5-8d68da77cd16 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.703406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a07e0-b57f-420d-b4d5-8d68da77cd16 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.704599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 411/829] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:33.705301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a07e0-b57f-420d-b4d5-8d68da77cd16 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:33.730470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.731403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.733986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.734310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.735160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=acfbe8c6a73d4b4992faba36c9881012) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:33.735705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.737070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98debd-46a7-4892-abbe-71b5c128c804 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.743767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 419/830] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/role_assignments?user.id=acfbe8c6a73d4b4992faba36c9881012&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:33.745057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.745339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.745643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.767608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:33.769127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:33.818596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:33.866599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a07e0-b57f-420d-b4d5-8d68da77cd16 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:33.877942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 420/831] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:33.911536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:33.912558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:33.912862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:33.913104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:33.921153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:33.922699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:33.935915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:33.936886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:33.937964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 412/832] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:33.942929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.043222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:34.050594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0adcd2-29af-43f6-a2a0-8ce0574bb547 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:34.086926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:34.087457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:34.087732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:34.087999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:34.088682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:34.088979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:34.090494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:34.107847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 421/833] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:34.113570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.157131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.244154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a6a6-98d2-48b3-962e-6ec9f1c4b6b0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:34.270658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56361f-a49f-4b72-a7fc-d7900bfae760 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:34.285056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:34.299001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:34.299944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:34.300229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:34.300508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/802cad51148347a189b9150bcf4fcf18/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:34.309765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:34.310601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:34.311529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:34.311902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:34.319565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=802cad51148347a189b9150bcf4fcf18, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:34.321411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:34.323300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:34.330332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:34.331285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:34.332591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 422/834] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:34.338224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.344094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 413/835] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:34 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/802cad51148347a189b9150bcf4fcf18/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:47:34.365237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:34.464958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a193a-dfeb-4be0-80c5-89051fb0257a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:34.506705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:34.507339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:34.507620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:34.507899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:34.508583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:34.508888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:34.510332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:34.520915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 423/836] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:34.531509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.646988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175b55b-e0c3-4566-bd0c-017b866d2b5a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:34.683223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:34.683864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:34.684146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:34.684423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:34.695250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:34.696698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:34.711744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:34.714015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:34.716118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:34.716118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:34.716118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:34.716118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 414/837] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:34.722634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.810687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:34.841191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dba10-efb8-4d0f-a4ae-65171456f4c5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:34.876753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:34.877365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:34.877683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:34.878049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:34.878687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:34.878990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:34.885816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:34.905124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 424/838] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:34.910637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:34.941260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:35.016062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87edb9e-279c-400c-b6a1-c5a676b7e3a1 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:35.050843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:35.051835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:35.052328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:35.052847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:35.060862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:35.062663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:35.074546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:35.075584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:47:35.077129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:35.079725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 415/839] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:47:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:35.085644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:35.191145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b61fa6-c097-4d21-9bf3-f92c73a8780d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:35.227290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:35.228250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:35.228832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:35.229340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:35.230290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:35.230839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:35.232523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:35.240946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:35.255408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 425/840] 10.222.0.23 () {56 vars in 1135 bytes} [Sat Sep 10 20:47:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:35.258365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:35.352178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:35.364067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57b073-7006-445d-a89f-e789da9b20b8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:35.398786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:35.399450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:35.399732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:35.400055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:35.469863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:35.471679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:35.561872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:35.575053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 416/841] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:47:35.581427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:35.692527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d8f97-f7e3-442c-8012-b3b418d14a0c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:35.729012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df309539-9546-4e60-93a2-96088a80101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:35.729601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df309539-9546-4e60-93a2-96088a80101f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:35.729886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df309539-9546-4e60-93a2-96088a80101f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:35.730158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df309539-9546-4e60-93a2-96088a80101f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:35.741182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df309539-9546-4e60-93a2-96088a80101f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:35.742674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df309539-9546-4e60-93a2-96088a80101f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-df309539-9546-4e60-93a2-96088a80101f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:35.757615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:35.758604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:35.759841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:35.759841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:35.759841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:35.759841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 426/842] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:35.765175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:35.869192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ab09b-d0bd-4651-83ae-8eb517c58a56 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:35.907608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:35.908228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:35.908504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:35.908781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:35.909430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:35.909762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:35.911116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cbadf-2788-41f6-a6b9-5620ec7a1b14 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:35.923033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 417/843] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:35.928624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df309539-9546-4e60-93a2-96088a80101f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:35.985932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:36.039453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df309539-9546-4e60-93a2-96088a80101f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:36.080544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:36.081214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.081525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.081808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.087032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d39bd-6997-48c1-a277-408f37841c79 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.087671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d39bd-6997-48c1-a277-408f37841c79 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.088156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d39bd-6997-48c1-a277-408f37841c79 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.090029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:36.092308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:36.092573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 418/844] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:36.097509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c4cd7-c54e-4e8e-8672-421369181706 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.097950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c4cd7-c54e-4e8e-8672-421369181706 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.098326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c4cd7-c54e-4e8e-8672-421369181706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:36.106019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:36.107009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:36.108445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 427/845] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:36.115273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.121361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-369c4cd7-c54e-4e8e-8672-421369181706 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:36.123711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-369c4cd7-c54e-4e8e-8672-421369181706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:36.177648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:36.209501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 419/846] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:36.225707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e79668-3cdc-49b4-bc93-dd95cc1802ed None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.226299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e79668-3cdc-49b4-bc93-dd95cc1802ed None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.226927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e79668-3cdc-49b4-bc93-dd95cc1802ed None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.229015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 420/847] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:36.235052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.235872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.236395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.243674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdac6f8-7601-477b-b636-1ec6bd94b00c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:36.253747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:36.256451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:36.257954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:36.286805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:36.288173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.288891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.289348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.290915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:36.291428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:36.293555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:36.313150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 428/848] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:36.318404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.353500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 421/849] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:36.431575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa78d0-c303-4548-aad3-0f62e12586a5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:36.468228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:36.469165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.469648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.470098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.471005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=233de9dbbb7a48159a1b0acdcd9880b3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:36.471469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:36.473109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38157e16-2cb9-4639-ba07-2081a5cd4980 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:36.480791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 429/850] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/role_assignments?user.id=233de9dbbb7a48159a1b0acdcd9880b3&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:36.502141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.606167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac6330-0bba-430b-ad86-213f5fea2e11 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.606711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac6330-0bba-430b-ad86-213f5fea2e11 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.606711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac6330-0bba-430b-ad86-213f5fea2e11 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.609161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 430/851] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:36.614833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89694325-ff95-4020-a479-4aae421300a3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.615140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89694325-ff95-4020-a479-4aae421300a3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.615393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89694325-ff95-4020-a479-4aae421300a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.626307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d8f27-01d9-4ad5-a9a5-dcbbdaee1243 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:36.634700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-89694325-ff95-4020-a479-4aae421300a3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:36.635923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-89694325-ff95-4020-a479-4aae421300a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:36.636854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:36.666333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:36.673623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:36.674539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.674819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.675091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.675686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:36.677070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:36.686256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:36.687310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:36.688148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 422/852] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:36.689023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.723480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 431/853] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:36.724098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89694325-ff95-4020-a479-4aae421300a3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.810248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0955c-844b-469f-8a54-11ec4f674a37 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:36.852124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:36.853086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.853371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.853916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.854909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:36.856566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:36.857562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89694325-ff95-4020-a479-4aae421300a3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:36.867651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:36.868602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:36.869499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:36.870247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 423/854] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:36.876303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.919374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:36.920896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:36.921188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:36.921492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:36.922551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:36.922864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:36.924627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:36.930995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:36.932784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 432/855] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:36.938996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:36.977005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:37.002858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d40a40-c0d2-4926-9d76-209f474bd21f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.042391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.043285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.043573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.043889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.044494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:37.045331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.046722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.053581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 424/856] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:37.053581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:37.062931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564361a-526f-4aa4-b567-e376962fbd30 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.100193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.101147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.101685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.102176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.103025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:37.103431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.104803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.121488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 433/857] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:37.121488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:37.183162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:37.188707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca333-e879-428a-bba6-fc16e3ba17e4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.227787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.228548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.228932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.229302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.234374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:37.242561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:37.247003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff443e5-f8c6-4563-a7e5-1ebc9a21490e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:37.247716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:37.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:37.250126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 425/858] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:37.255790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:37.302708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.303527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.303911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.304292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.305657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:37.306075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.308434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.316812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 434/859] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:37.381521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b66f75-3026-4fd0-af5a-266b9aecb5c2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.424902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.425617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.425905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.426185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.426901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:37.427213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.428745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.440040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 426/860] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:37.446838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:37.560837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bf861-2256-4b44-890f-72b73ec64cc2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.598826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.599375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.599651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.599921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.611158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.612696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:37.627998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:37.628958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:37.629870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:37.629870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:37.629870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:37.633632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 435/861] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:37.635178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:37.663905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:37.751176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74119ed0-2f47-4165-92d9-f5d0046e3c74 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.787155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.787781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.788062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.788333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.788978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:37.789286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.790692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.807989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 427/862] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:37.813621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:37.823484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:37.928632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bd7b9-6c75-497a-9f76-e644d6d27c98 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:37.964098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:37.964729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:37.965004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:37.965274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:37.973873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:37.975309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:37.987751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:37.988717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:37.989684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:37.993579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 436/863] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:37.994927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:38.083078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:38.103181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731410c-d680-4565-a841-7bd6c64af855 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:38.139933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:38.140806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:38.141079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:38.141345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:38.142148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:38.142458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:38.143867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:38.168287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 428/864] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Sep 10 20:47:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:38.169803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:38.240703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:38.280858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91ec4f-aedd-470c-bd98-8cf52902f031 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:38.315614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:38.316221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:38.316505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:38.316779 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:38.392729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:38.394269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:38.462292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:38.500408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 437/865] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:47:38.505776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:38.609043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8234-7faf-4c28-9222-71c4de2df0db None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:38.646596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:38.647259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:38.647519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:38.647792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:38.658385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:38.660550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:38.662438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-0dabc276-4376-4e6a-ac31-c979620f6998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:38.679631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:38.680645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:38.681594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:38.681594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:38.681594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:38.685591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 429/866] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:38.690546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:38.772106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f056-b416-449b-a418-51a29a96eee3 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:38.772315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f056-b416-449b-a418-51a29a96eee3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:38.772560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675f056-b416-449b-a418-51a29a96eee3 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:38.774689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 430/867] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:38.783201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:38.783591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:38.784128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:38.798651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e08b8-f948-4cc2-9b51-eb343269405d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:38.811469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:38.812956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:38.834941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:38.835543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:38.835819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:38.836085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:38.836707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:38.837109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:38.838538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:38.851315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 438/868] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:38.856769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:38.892623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:38.903634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 431/869] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:38.976941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8f59-2381-4f1a-8e68-91da6f25e8e6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:39.013532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:39.014512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.015029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.015782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.023159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:39.024869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:39.037272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:39.038213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:39.039087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:39.041404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 439/870] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:39.050564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:39.155573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12013956-26a0-4eb9-a94a-ada557ff4437 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:39.179499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:39.181757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:39.192274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:39.192890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.193170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.193442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.194159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:39.194461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:39.195887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:39.213082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 432/871] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:39.214350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:39.280451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:39.304525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4981ca-9ecb-4628-98b0-2f7953931a8b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:39.329676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07b31-99e9-4440-a501-ccac62a98efb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:39.350957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:39.351982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.352491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.352965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.353839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:39.355493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:39.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:39.367983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:39.368837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:39.377147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:39.377147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.377147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.377147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.377576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=802cad51148347a189b9150bcf4fcf18) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:39.377576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:39.377967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 440/872] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:39.378773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0850ed8-c6da-49ad-be62-945be7f75439 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:39.379386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:39.393653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 433/873] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:39 2022] GET /identity/v3/role_assignments?user.id=802cad51148347a189b9150bcf4fcf18&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:39.473921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7bd98-0392-49e1-ba90-2f13d3311a6f None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.474562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7bd98-0392-49e1-ba90-2f13d3311a6f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.475276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7bd98-0392-49e1-ba90-2f13d3311a6f None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.478032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 434/874] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:39.485146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05228d1a-9964-4518-a484-f7fd17ae49e4 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.485556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05228d1a-9964-4518-a484-f7fd17ae49e4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.485753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05228d1a-9964-4518-a484-f7fd17ae49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.503533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55800a45-03c9-4c64-9dd1-b5527a5ec92c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:39.505893 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-05228d1a-9964-4518-a484-f7fd17ae49e4 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:39.507182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-05228d1a-9964-4518-a484-f7fd17ae49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:39.556152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:39.556790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.557187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.557489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.558213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:39.558523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:39.559946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:39.564766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:39.568597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 441/875] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:39.574735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:39.605487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:39.605681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 435/876] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:39.618818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40766cc0-3a7c-4f42-9c15-c4b74af25cfa None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.619099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40766cc0-3a7c-4f42-9c15-c4b74af25cfa None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.619380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40766cc0-3a7c-4f42-9c15-c4b74af25cfa None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.620835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 436/877] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:39.626909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.627192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.627468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.647576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:39.648779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:39.709250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5625-0d2e-4324-a011-158d8934667c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:39.740727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 437/878] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:39.754447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:39.755422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:39.755958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:39.756443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:39.757260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:39.757834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:39.759439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:39.764956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 442/879] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:39.878716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:39.992850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc159c8-8e24-4c05-8413-627a53f7fb67 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:40.001070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:40.022485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:40.034598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b0a02-3e97-428b-b4c0-bc6d7cf2d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:40.035564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b0a02-3e97-428b-b4c0-bc6d7cf2d02e None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:40.036013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b0a02-3e97-428b-b4c0-bc6d7cf2d02e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:40.036418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b0a02-3e97-428b-b4c0-bc6d7cf2d02e None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:40.037090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b0a02-3e97-428b-b4c0-bc6d7cf2d02e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:40.038504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b0a02-3e97-428b-b4c0-bc6d7cf2d02e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:40.058090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 438/880] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:39 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:40.113446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:40.143516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7afa2-e328-4b89-9420-f022e0030db2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:40.191031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298eaa4-f3b9-4d01-998d-864ae00f7b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:40.191667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298eaa4-f3b9-4d01-998d-864ae00f7b1d None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:40.191942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298eaa4-f3b9-4d01-998d-864ae00f7b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:40.192219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298eaa4-f3b9-4d01-998d-864ae00f7b1d None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:40.192897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298eaa4-f3b9-4d01-998d-864ae00f7b1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:40.194324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298eaa4-f3b9-4d01-998d-864ae00f7b1d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:40.214166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 443/881] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:40 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:47:41.786092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0622d3-4bfe-48b4-8c52-834feaf3b312 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:41.786750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0622d3-4bfe-48b4-8c52-834feaf3b312 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:41.786750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0622d3-4bfe-48b4-8c52-834feaf3b312 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:41.788410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 439/882] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:41.793602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5be2-b481-4bf3-a552-c22a19417168 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:41.794416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5be2-b481-4bf3-a552-c22a19417168 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:41.795487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5be2-b481-4bf3-a552-c22a19417168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:41.814253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b0ae5be2-b481-4bf3-a552-c22a19417168 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:41.815796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b0ae5be2-b481-4bf3-a552-c22a19417168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:41.901571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 444/883] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:42.055710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf6d4b-52e1-4704-8f1e-38cdecdc2ad3 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.055710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf6d4b-52e1-4704-8f1e-38cdecdc2ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.055710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf6d4b-52e1-4704-8f1e-38cdecdc2ad3 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.056288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 440/884] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:42.062111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8894e-620b-44c9-96a6-e7fbef25cdec None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.062641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8894e-620b-44c9-96a6-e7fbef25cdec None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.063121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8894e-620b-44c9-96a6-e7fbef25cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.081899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-56e8894e-620b-44c9-96a6-e7fbef25cdec None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:42.083420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-56e8894e-620b-44c9-96a6-e7fbef25cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:42.141845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:42.156461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 445/885] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:42.172465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf6d4b-52e1-4704-8f1e-38cdecdc2ad3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.285556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf6d4b-52e1-4704-8f1e-38cdecdc2ad3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:42.304151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104648d-b716-4c81-a8fa-e717a9049ac8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.304453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104648d-b716-4c81-a8fa-e717a9049ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.304712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104648d-b716-4c81-a8fa-e717a9049ac8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.306383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 446/886] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:42.314323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.314869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.315489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.320887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:42.321559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.321846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.322121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.334028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:42.335973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:42.337352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:42.338909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:42.339808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:42.340643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 441/887] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:42.345842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.376111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:42.433800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 447/888] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:42.443519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.463949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f1d64-929c-4cfe-b701-2f45db1610d1 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:42.503638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:42.504266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.504552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.504828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.505629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:42.505942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:42.507401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b0134-6252-4683-8d11-a2e9d9927cc5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:42.518425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 442/889] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:42.526970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2cfc-78cc-44da-bbdf-84006414bcaf None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.527265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2cfc-78cc-44da-bbdf-84006414bcaf None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.527546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2cfc-78cc-44da-bbdf-84006414bcaf None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.528866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 443/890] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:42.530383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2cfc-78cc-44da-bbdf-84006414bcaf None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.550359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd70bd-4557-43e0-b113-3b57ee53e386 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:42.574454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:42.590031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa72d5-d18f-449c-b13b-c5200c4f1912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:42.590639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa72d5-d18f-449c-b13b-c5200c4f1912 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.590920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa72d5-d18f-449c-b13b-c5200c4f1912 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.591194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa72d5-d18f-449c-b13b-c5200c4f1912 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.592576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa72d5-d18f-449c-b13b-c5200c4f1912 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:42.593853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa72d5-d18f-449c-b13b-c5200c4f1912 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:42.609108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 448/891] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:42 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:47:42.610614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.611163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.611616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.640206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:42.642128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:42.643184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:42.666054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2cfc-78cc-44da-bbdf-84006414bcaf None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:42.712693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:42.713872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.714230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.714690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.730388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:42.731841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:42.749925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 449/892] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:42.749925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:42.750190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:42.751083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:42.751983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:42.751983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:42.751983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:42.754735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 444/893] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:42.756082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.871421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b204a6c-5b2e-4dab-a5f9-7092c64859f0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:42.877058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adcd28-fd32-4d16-9aa6-d02c9a4dc736 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:42.890247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:42.925271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:42.925937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.926412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.926685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.927265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:42.928624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:42.936542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:42.937364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:42.937895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:42.938310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:42.939097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:42.939548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:42.941039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:42.941440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:42.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:42.943402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 450/894] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:42.947680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:42.950064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:42.965854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 445/895] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:42.967775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.083138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2aa82-37ee-42a2-9ba4-cad18140fc61 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.086898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce596f7c-df3e-44f8-b4da-2bc50c6233d7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.120377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.120984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.121257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.121609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.122178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.122478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.123956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.124590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.124688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.124869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.125141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.134269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 451/896] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:43.134269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.134482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.134482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:43.146766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:43.147925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:43.148877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 446/897] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:47:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:43.149875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.182033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.210821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.246117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816d6bf-5024-42aa-9273-8e2ccffd1091 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.258378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6052dae-06d5-4a76-8d59-e5977b86ac63 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.282175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.282798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.283074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.283343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.283913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.285317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:43.295311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:43.296281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:43.297154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 452/898] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:43.298076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.299255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.299864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.300142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.300410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.301036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.301337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.302722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.312944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 447/899] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:43.312944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.411082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5449-b837-4668-8ec0-715c3e189788 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.412664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.426023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f0af6-e31b-45ba-a72a-c6d1de22f821 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.448414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.448704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.449313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.449675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.449886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.450752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.451044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.453035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.464666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.465281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.465595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.465874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.467209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.467529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.468897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.475864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 453/900] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:43.475864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.481518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 448/901] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:43.481518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.580747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93ba9-73d3-4c6d-8214-d46ccd3f3f52 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.590068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49c3ef-1695-4262-a454-63b635902f3d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.618104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.618863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.619145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.619420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.620107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] RBAC: Adding query filter params (interface=public, service_id=062f2528be134040a89d9148d8b1f384, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.620414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.621798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.627250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.627852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.628134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.628404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.633946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 454/902] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/endpoints?service_id=062f2528be134040a89d9148d8b1f384&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:43.633946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.661517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.688392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.705756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.707362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.744553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46de329-e4f4-467b-9fe1-3f542f504c86 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.780922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.781657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.781929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.782206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.782824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.783128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.784574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.789456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 455/903] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:43.790255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.822319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 449/904] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:47:43.824207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.887020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.902325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553ff81-8dac-4e99-8397-b349cecf9ade None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.925397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:43.942109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.942721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.942989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.943260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.943780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb691-f565-4863-963c-4be0321c326e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:43.943885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.945205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:43.952032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 456/905] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:47:43.952827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:43.990912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:43.991588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:43.991871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:43.992147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:43.992869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] RBAC: Adding query filter params (interface=public, service_id=06f526832cf1472e8785c7e664aa83a3, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:43.993176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:43.994765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.005522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 450/906] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/endpoints?service_id=06f526832cf1472e8785c7e664aa83a3&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:44.007203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:44.096842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274afcca-dc23-4b61-900a-656f0d650a81 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:44.135062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:44.135708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.135986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.136261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.144162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d4ca4-84bc-4995-9717-e4a3b8ab358d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:44.155900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:44.155900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:44.173542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:44.174528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:44.175389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:44.175389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:44.175389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:44.175389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 457/907] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:44.180435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:44.185642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:44.191887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:44.193931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.194350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.194766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.195980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:44.198085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.206115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 451/908] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:44 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:47:44.298427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552574a2-c686-45ec-ba6a-d249910b9dc4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:44.336734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:44.337643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.338065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.338513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.339328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:44.339787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:44.342039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.356382 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 458/909] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:44.361681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:44.378468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:44.475459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f3cb-5cd6-431b-a654-d0bbd87fba30 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:44.512703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:44.513432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.513983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.514362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.521480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:44.522978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:44.534562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:44.535665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:44.536732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 452/910] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:44.541692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:44.611460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:44.650974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367d0ac-cddb-422f-801c-deda88a89133 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:44.687551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:44.688402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.688822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.689261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.690128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:44.690572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:44.692122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae40-f4fc-4c01-bdf6-496afc26c345 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.705100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 459/911] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:44.710905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:44.790465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:44.828718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc677194-403a-4a9a-ad8f-ae37f198c7da None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:44.864550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:44.865433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.865921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.866364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.867226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] RBAC: Adding query filter params (role.id=860aa35e370b4ccdb3d60a5b458123e7, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=802cad51148347a189b9150bcf4fcf18) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:44.867676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:44.869238 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0be9f-4b46-412a-9544-d182d569cc80 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:44.875749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 453/912] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:44 2022] GET /identity/v3/role_assignments?user.id=802cad51148347a189b9150bcf4fcf18&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=860aa35e370b4ccdb3d60a5b458123e7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:44.893358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1eae9-a58a-4aec-ba54-6e137b4b73c7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.894257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1eae9-a58a-4aec-ba54-6e137b4b73c7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.894679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1eae9-a58a-4aec-ba54-6e137b4b73c7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.896330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 460/913] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:44.904706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:44.905232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:44.905698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:44.925790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:44.927896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:45.007211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 454/914] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:45.276782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1eae9-a58a-4aec-ba54-6e137b4b73c7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:45.386592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1eae9-a58a-4aec-ba54-6e137b4b73c7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:45.417835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:45.423671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:45.424595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:45.425174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:45.425694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:45.426626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:45.428001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:45.438054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:45.439031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:45.439870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 461/915] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:45.446850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:45.511616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:45.548475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80414edd-6d5e-4f02-bee7-d368c218201c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:45.587783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:45.588404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:45.588690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:45.588969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:45.589637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:45.589946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:45.591417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:45.596618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 455/916] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:45.602551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:45.709250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ef5-148a-4b37-9a73-bc3a87ae2aa8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:45.746444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:45.747117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:45.747444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:45.747728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:45.748487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:45.748788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:45.750191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:45.761012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 462/917] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:45.762449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:45.873497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91e3d2-07a5-4455-94da-253953ead0d1 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:45.905803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:45.910290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:45.910908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:45.911191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:45.911466 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:45.912198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] RBAC: Adding query filter params (interface=public, service_id=0f6dbb8c34944fe99e0a3137ce51f672, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:45.912514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:45.914002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d29e7-96c7-4dd0-b74d-0d8a55cdf9b1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:45.919683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 456/918] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:47:45 2022] GET /identity/v3/endpoints?service_id=0f6dbb8c34944fe99e0a3137ce51f672&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:45.925374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:45.984408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:46.034410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0db6d-5523-428c-953f-619e03755a1c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:46.073490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61688a1-2c2d-454c-8595-24299d47a256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:46.074111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61688a1-2c2d-454c-8595-24299d47a256 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:46.074394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61688a1-2c2d-454c-8595-24299d47a256 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:46.074670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61688a1-2c2d-454c-8595-24299d47a256 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:46.075275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61688a1-2c2d-454c-8595-24299d47a256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:46.076698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61688a1-2c2d-454c-8595-24299d47a256 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:46.083438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 463/919] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:45 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:47:46.295981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903845-249f-4bb2-8350-2293e2e224d8 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:46.296983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903845-249f-4bb2-8350-2293e2e224d8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:46.297510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903845-249f-4bb2-8350-2293e2e224d8 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:46.299337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 457/920] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:46.305794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:46.306396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:46.306876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:46.328872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:46.330446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:46.416917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 464/921] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:46.726405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272d221-2456-4fc5-bf57-63930e84dd3e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:46.726705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272d221-2456-4fc5-bf57-63930e84dd3e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:46.726889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272d221-2456-4fc5-bf57-63930e84dd3e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:46.728512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 458/922] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:46.734591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:46.736195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:46.736548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:46.736833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:46.757510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:46.759431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:46.813857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:46.864896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 459/923] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:46.884617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d658-8a06-41e4-a8c7-30279d3778fc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:46.926987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:46.927635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:46.927894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:46.928169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:46.931165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:46.933684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:46.945873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:46.946900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:46.947807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 465/924] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:46.952396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.075285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ed7e5-313f-4dac-9662-51249ead7e6f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.103501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:47.119055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.119676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.119998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.120344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.121427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:47.121979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:47.124198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:47.129554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 460/925] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:47.138800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.200147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.294443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b7d5-3155-4625-b8b4-83563688596a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.306542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:47.336106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4e60-af29-4a52-a828-4598f2f70b4c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.337974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.338581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.338865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.339133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.339760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:47.340057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:47.342020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affd6a8-d83c-4438-99dd-325e9e3a4edc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:47.347042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 466/926] 10.222.0.23 () {56 vars in 1114 bytes} [Sat Sep 10 20:47:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:47.348221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0490-dd76-432b-aabb-9b3cdb63fdd7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.348500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0490-dd76-432b-aabb-9b3cdb63fdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.348768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0490-dd76-432b-aabb-9b3cdb63fdd7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.350155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 467/927] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:47.353087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a0490-dd76-432b-aabb-9b3cdb63fdd7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.380762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.381404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.381714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.381997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.382579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:47.383959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-5ae75cda-6ddc-482d-82b7-e199def3d06e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:47.393223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:47.399325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:47.400282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 461/928] 10.222.0.23 () {56 vars in 1132 bytes} [Sat Sep 10 20:47:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:47.401106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.401106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.401106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.417062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:47.418399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:47.473000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:47.484203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a0490-dd76-432b-aabb-9b3cdb63fdd7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.498772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 462/929] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:47.501928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.530272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.530888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.531225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.531504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.532408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:47.533570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:47.565498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 468/930] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:47:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:47.639016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8dede-81a1-4333-9364-fdbc3e5862d6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.687008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.687760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.688048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.688378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.689033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:47.689339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:47.690943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:47.696929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 463/931] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:47.703253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.739141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:47.770983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.826166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:47.829697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae104a-205c-4954-8bfc-b608ba390df4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.869214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.870131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.870557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.870987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.871755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:47.872199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:47.874379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:47.880163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 469/932] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:47.885803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:47.904475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75d70-9d80-4f68-8f89-a28ce8815cd5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:47.952094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:47.952979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:47.953403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:47.953843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:47.967343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:47.972673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:47.973623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:47:47.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:47.977188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 464/933] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Sep 10 20:47:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:47.983209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.000726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:48.009528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a0586-37cc-4ef3-8853-00fb0b845ece None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.046550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.047268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.047622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.047960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.048698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.050232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:48.077009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 470/934] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:47:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:48.108886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04c05d-001e-4594-9f47-af8c75c39967 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.111061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:48.160846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.161949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.162407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.162854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.163839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:48.164319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.166036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d0dbd-d78d-4356-b473-b40ab87a5a67 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:48.176322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 465/935] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:47:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:48.182569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.309264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca60958-15d9-4cce-98a8-0e97705ffaee None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.345858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.346556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.346847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.347124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.359345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.360884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:48.376225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:48.377214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:48.378189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:48.378189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:48.378189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:48.378189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 471/936] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:48.384397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.384547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe3f0-cab6-41b0-bec9-83a0a862bd40 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.384955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe3f0-cab6-41b0-bec9-83a0a862bd40 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.385081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe3f0-cab6-41b0-bec9-83a0a862bd40 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.386677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 466/937] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:48.387878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:48.403091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.404058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.404736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.428425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:48.430031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:48.500713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7a8b7-2bc1-4f72-af74-e5da80ac8227 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.513494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 467/938] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:48.543864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.545104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.545451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.545756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.546403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:48.546706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.548131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:48.562624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 472/939] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:48.568447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.666671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:48.677049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe02d-75fc-434f-98d5-55e404f58aeb None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.715043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.715609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.715884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.716183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.723486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.724904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:48.736464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:48.737547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:48.738504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 468/940] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:47:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:48.743503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.809399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:48.817416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.855199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0251c7-35e9-4ec4-bb87-3aefc79c6ea6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.893694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.894387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.894639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.894981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.895660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:48.895967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.897415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:48.924392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 473/941] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Sep 10 20:47:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:48.925581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:48.948399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53d979-9fe3-414e-bc3e-fcf23b1603c0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:48.991097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:48.991661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:48.991944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:48.992219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:48.992804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:48.994223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:49.006733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:49.007748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:49.008586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:49.012954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 469/942] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:49.013864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.034777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:49.049841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68271dc3-2b74-4f96-9ac6-02a30e9ae7df None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.078305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:49.098136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.099333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.099811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.100853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.115686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.118091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.139248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6012fe-9999-475a-8d93-1b4e9c50096f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:49.140209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:49.141368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:49.142302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:49.142302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:49.142302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:49.145780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 474/943] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:49.149092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.189577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625e832-0f19-428e-841e-432767882d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.190375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625e832-0f19-428e-841e-432767882d7d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.190663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625e832-0f19-428e-841e-432767882d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.190938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625e832-0f19-428e-841e-432767882d7d None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.191923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625e832-0f19-428e-841e-432767882d7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:49.192235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625e832-0f19-428e-841e-432767882d7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.193961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625e832-0f19-428e-841e-432767882d7d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.203988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 470/944] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:49.204837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625e832-0f19-428e-841e-432767882d7d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.280911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b3a30-c2fa-4899-8038-8fb781ecff60 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.303646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:49.318347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.318865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.319211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.319496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.320139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:49.320451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.321856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.334640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625e832-0f19-428e-841e-432767882d7d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.340417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 475/945] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:49.341538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.373654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.374249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.374585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.374863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.375549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:49.375862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.377478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.380958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:49.389347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 471/946] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:49.391202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.460353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92216e4-2262-4f01-85a1-cea9efd316c3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.495138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.495772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.496047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.496312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.505344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.505952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e7cdd-34f2-43b2-a831-fc6b75027067 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.506831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:49.531223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:49.532192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:49.533104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:49.534758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 476/947] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:49.541976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.550881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.551501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.551780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.552053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.552718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.554122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.570805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:49.580126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 472/948] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:47:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:49.698661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:49.709192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53983b-5834-4248-8438-d594e224fbdc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.755846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.756471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.756749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.757018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.757815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:49.758118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.759584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2290c-c9a8-4104-888d-ea05befcf477 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.775321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 477/949] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:49.781813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:49.888905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ef220-9d85-4f17-b78d-1a36228f81e6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:49.903269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494fe7-cffd-4382-a119-68e26706d762 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.903725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494fe7-cffd-4382-a119-68e26706d762 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.904176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494fe7-cffd-4382-a119-68e26706d762 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.906055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 478/950] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:49.911316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc885826-4c43-4a7e-b7fa-d31ad9490dd0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.911604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc885826-4c43-4a7e-b7fa-d31ad9490dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.911885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc885826-4c43-4a7e-b7fa-d31ad9490dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.931864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:49.932543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:49.932669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fc885826-4c43-4a7e-b7fa-d31ad9490dd0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:49.932858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:49.933097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/802cad51148347a189b9150bcf4fcf18/roles/860aa35e370b4ccdb3d60a5b458123e7` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:49.933893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fc885826-4c43-4a7e-b7fa-d31ad9490dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:49.935469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=802cad51148347a189b9150bcf4fcf18, role_id=860aa35e370b4ccdb3d60a5b458123e7)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:49.937059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:49.955406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 473/951] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:47:49 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/802cad51148347a189b9150bcf4fcf18/roles/860aa35e370b4ccdb3d60a5b458123e7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:47:50.023317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 479/952] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:50.316030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:50.383394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:50.437550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427df0-8ac3-4177-bd37-5fee30892914 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:50.474609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:50.475229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:50.475510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:50.475786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:50.476509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:50.477988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:50.480605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING keystone.server.flask.common [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 10 20:47:50.500899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.sql.core [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 20:47:50.500899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 20:47:50.500899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [parameters: {'id': '3a53cd53740e4338b0d609cd10a451fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 20:47:50.500899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81440) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 20:47:50.504609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda6f61-85e2-4de3-b943-d5c7901672f1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:50.505046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda6f61-85e2-4de3-b943-d5c7901672f1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:50.505212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda6f61-85e2-4de3-b943-d5c7901672f1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:50.506765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 480/953] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:50.511745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:50.512031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:50.512301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3a53cd53740e4338b0d609cd10a451fe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3a53cd53740e4338b0d609cd10a451fe. Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:47:50.521007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application result.read() Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 10 20:47:50.521923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application yield resource Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise self.value Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application session.commit() Sep 10 20:47:50.522892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self.session.flush() Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application compat.raise_( Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 20:47:50.523795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 20:47:50.524725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application result.read() Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:47:50.526540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [parameters: {'id': '3a53cd53740e4338b0d609cd10a451fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:50.529300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:50.538065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3a53cd53740e4338b0d609cd10a451fe. Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 474/954] 10.222.0.23 () {60 vars in 1140 bytes} [Sat Sep 10 20:47:50 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:50.539303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:50.540050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:50.540050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:50.632859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 481/955] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:50.660885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760382ad-e170-47f0-b883-38679462ab66 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:50.675548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:50.699043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:50.699679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:50.699954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:50.700232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:50.707869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:50.709660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:50.722022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:50.724105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:50.726334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:50.733484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 475/956] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:50.733694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:50.744841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:50.835468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37f5ca-b355-43fd-859c-545d36ae15a8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:50.870475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:50.871086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:50.871364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:50.871637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:50.872429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:50.872728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:50.874177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:50.898588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 482/957] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:50.927997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:51.039924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7a544-f6ef-4fe5-8492-6b7882f5212d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:51.077378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:51.078059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:51.078340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:51.078616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:51.079206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:51.080574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:51.089662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:51.092681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:51.093759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 476/958] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:51.097366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:51.169980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:51.207607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20674fe2-d240-4ce2-90d1-dd4271f7f936 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:51.242257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:51.242933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:51.243218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:51.243493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:51.244120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:51.244422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:51.245842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd7765-a28b-48e3-8b9a-8b6f57c5955c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:51.252074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 483/959] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:51.257730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:51.301179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:51.364777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cdba7-8868-49b3-bef9-e0fda0bf467d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:51.401915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:51.403017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:51.403299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:51.403640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:51.404267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:51.404573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:51.405984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786c76-452f-4a27-b462-1cfcfd22bdd8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:51.414035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 477/960] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:51.864877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bbc07-40ae-4205-b280-95f62accd4b9 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:51.865624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bbc07-40ae-4205-b280-95f62accd4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:51.865624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bbc07-40ae-4205-b280-95f62accd4b9 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:51.867046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 484/961] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:51.873283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b0f39-a62f-499c-8cd0-0ddd18acadd1 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:51.873828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b0f39-a62f-499c-8cd0-0ddd18acadd1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:51.874027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b0f39-a62f-499c-8cd0-0ddd18acadd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:51.895912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-789b0f39-a62f-499c-8cd0-0ddd18acadd1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:51.895912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-789b0f39-a62f-499c-8cd0-0ddd18acadd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:51.969136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 478/962] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:52.218044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826701-1023-4aac-8f66-d6770001823c None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.218407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826701-1023-4aac-8f66-d6770001823c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.218592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826701-1023-4aac-8f66-d6770001823c None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.220075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 485/963] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:52.225253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.225573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.225855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.245186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:52.246674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:52.251418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826701-1023-4aac-8f66-d6770001823c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.278101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:52.329489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 479/964] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:52.375659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826701-1023-4aac-8f66-d6770001823c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:52.406917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:52.413411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:52.414073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.414353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.414629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.415210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:52.416576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:52.426404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:52.427404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:52.428238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 486/965] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:52.433710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.540409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fe0bb-408d-46ea-aa86-fd910ad62aad None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:52.579145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:52.580131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.580609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.581090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.582145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:52.582647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:52.584366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:52.593627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 480/966] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:52.598980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.620271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.661360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:52.718689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f2400-c9aa-411c-a22a-6e55523f5954 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:52.755286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48dd1-84b9-4b7e-b9cf-8f4975e6f9b3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:52.759021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:52.759862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.760338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.760834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.761707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:52.762229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:52.764583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:52.770411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 487/967] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:52.776203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.801308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:52.801921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.802210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.802490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.816011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:47:52.821174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:52.822148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:47:52.822983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:52.830906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 481/968] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:52.832533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.845534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:52.897326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc3d34-599e-4637-a0a9-b05efc782a0d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:52.935315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:52.935594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:52.936063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:52.936483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:52.937270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] RBAC: Adding query filter params (interface=admin, service_id=0f6dbb8c34944fe99e0a3137ce51f672, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:52.937690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:52.939153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:52.947030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:52.949647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 488/969] 10.222.0.23 () {56 vars in 1249 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/endpoints?service_id=0f6dbb8c34944fe99e0a3137ce51f672&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:52.951071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:52.965503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da7925-1b58-4056-90da-4afdf4a9f246 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.001908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.002907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.003400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.003870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.004718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:53.005217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.006801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.021396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 482/970] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:53.022616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:53.060297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6d18f-45a1-43eb-9c18-88cdbfc43459 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.086150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:53.099469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.099892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.100170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.100443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.101053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.102480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.110057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 489/971] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:52 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:47:53.134850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c52d1-18ec-43e1-8eb1-c99d48cd589c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.175023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.176045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.176537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.177003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.190690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.192512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.203195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-8212b2a9-a514-45b5-8de3-072af031e44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:53.211308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:53.212386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:53.213407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:53.213407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:53.213407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:53.216463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 483/972] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:53.222994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:53.337587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ff70d-83dc-4971-a832-e5e9240eda35 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.339286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ad3e-ca73-4302-8959-846b260b682a None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.339814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ad3e-ca73-4302-8959-846b260b682a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.340273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ad3e-ca73-4302-8959-846b260b682a None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.342925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 484/973] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:53.353495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.353495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.353495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.372895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:53.374370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:53.383614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.384233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.384510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.384780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.385418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:53.386136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.387902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.404094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 490/974] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:53.409451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:53.460158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 485/975] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:53.473871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:53.530733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329b981-a678-4bb3-827b-9cc46d05d095 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.571403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.572320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.572764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.573206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.581899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.583715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e20fed52-788a-4bd7-b9cb-dc8166f96113 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:53.595718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:53.596868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:47:53.598107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:53.600262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 491/976] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:47:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:53.605819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:53.646257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e5709-7b22-4471-a722-a8005a29e6ce None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.646539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e5709-7b22-4471-a722-a8005a29e6ce None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.646821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e5709-7b22-4471-a722-a8005a29e6ce None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.648427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 492/977] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:47:53.653285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84719422-c9fa-471c-9275-c360c243f08c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.653664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84719422-c9fa-471c-9275-c360c243f08c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.653949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84719422-c9fa-471c-9275-c360c243f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.673170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-84719422-c9fa-471c-9275-c360c243f08c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:53.674521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-84719422-c9fa-471c-9275-c360c243f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:53.715459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969ce9c-88c3-4f6d-ab12-56354941acd7 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.749783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:53.757492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.761488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.761488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.761488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.761488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:53.761488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.761488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.762000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 493/978] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:53.781184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 486/979] 10.222.0.23 () {56 vars in 1149 bytes} [Sat Sep 10 20:47:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:53.783103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84719422-c9fa-471c-9275-c360c243f08c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:53.786834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:53.829380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:53.901540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84719422-c9fa-471c-9275-c360c243f08c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.908140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4db0eb-7c52-4f7f-9503-bd727eb7c2ea None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:53.939529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.940310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.940691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.941084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.941894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:53.943653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:53.946343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING keystone.server.flask.common [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 10 20:47:53.948336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:53.949484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:53.949909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:53.950675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:53.961223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.sql.core [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 20:47:53.961223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 20:47:53.961223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [parameters: {'user_id': '67970e90b9074c21a0727686a104984d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 20:47:53.961223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Sep 10 20:47:53.969031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:47:53.969946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application yield resource Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise self.value Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application session.commit() Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 20:47:53.970886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.flush() Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application compat.raise_( Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 20:47:53.971763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:47:53.973334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:47:53.974287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [parameters: {'user_id': '67970e90b9074c21a0727686a104984d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:53.975515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 10 20:47:53.977368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 494/980] 10.222.0.23 () {60 vars in 1131 bytes} [Sat Sep 10 20:47:53 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:53.979631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.024859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:54.039740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.041534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.083890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:54.087472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d8f81-9d3d-4848-8468-5592a7f2c8be None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:54.127312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:54.127968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:54.128250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:54.128524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:54.136223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.138571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:47:54.150934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:54.151859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 495/981] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:47:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:54.152808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.161939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 487/982] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:47:53 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:47:54.161939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.252695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:54.265589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571678-b742-49bd-a2dd-3ad0accf5e44 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:54.268252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a4d52-fd7a-4893-a0fa-0e6fa53d7cd1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:54.303533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:54.303720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:54.304147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:54.304252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:54.304429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:54.304529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:54.304702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:54.304809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:54.305350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.305446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:54.305702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.306798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.307090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.333518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 496/983] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:54 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:54.333518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.357666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 488/984] 10.222.0.23 () {56 vars in 1101 bytes} [Sat Sep 10 20:47:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:54.362504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:54.474733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3ef0-8225-4018-85ea-ba213e142a80 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:54.520202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:54.521197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:54.521682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:54.522129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:54.535260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.537046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:47:54.553911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:54.554998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:47:54.555902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:47:54.555902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:47:54.555902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:54.557146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 497/985] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:54.563513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.671919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1606a47-a883-466f-9023-2d6740bb68f2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:54.707207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:54.707827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:54.708111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:54.708389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:54.709034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:54.709339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.710739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.724719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 489/986] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:54.730359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.800864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:54.836316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e215a1-fbcf-4763-804f-7607faa5f580 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:54.872306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:54.872946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:54.873220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:54.873514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:54.882214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:54.884632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:47:54.898192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:54.904948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:47:54.905909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 498/987] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:47:54.906399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:54.915995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:55.010931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2e780-0947-49b9-9302-a5f7216135a7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:55.045583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:55.046206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:55.046487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:55.046761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:55.047449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:55.047756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:55.049183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae256aa-96ba-47d7-af4c-98f37123d739 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:55.061544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 490/988] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:47:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:55.067276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:55.169843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5439-0b91-49f0-ba94-56c134908718 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:55.196184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:55.205183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:55.205840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:55.206115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:55.206385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:55.207108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] RBAC: Adding query filter params (role.id=860aa35e370b4ccdb3d60a5b458123e7, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=802cad51148347a189b9150bcf4fcf18) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:55.207414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:55.208834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda3fac-2c22-4838-8fef-bce7a2fc35ed None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:55.214829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 499/989] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:47:55 2022] GET /identity/v3/role_assignments?user.id=802cad51148347a189b9150bcf4fcf18&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=860aa35e370b4ccdb3d60a5b458123e7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:55.496486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14585f-1597-4e07-b3c3-c51a74a47fbd None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:55.496814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14585f-1597-4e07-b3c3-c51a74a47fbd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:55.497140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14585f-1597-4e07-b3c3-c51a74a47fbd None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:55.499319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 491/990] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:55.504640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:55.504931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:55.505211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:55.524234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:55.525768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:55.596589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 500/991] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:55.865233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14585f-1597-4e07-b3c3-c51a74a47fbd None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:55.943611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:55.981243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14585f-1597-4e07-b3c3-c51a74a47fbd None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:56.015956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:56.016632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:56.016914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:56.017191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:56.017793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:56.019152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:56.027984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:56.033763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:56.034678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 492/992] 10.222.0.23 () {56 vars in 1126 bytes} [Sat Sep 10 20:47:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:56.036867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:56.150140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736425a-9abc-47c3-a8fc-ca69c6bb7e2d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:56.174879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:56.188646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:56.189263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:56.189573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:56.189849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:56.190488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:56.190811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:56.192159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:56.197954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 501/993] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:56.204913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:56.311480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf781591-cf52-44aa-bb1c-6830a573b2ee None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:56.339709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:56.347991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:56.348606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:56.348886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:56.349226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:56.349891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:56.350199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:56.351566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27fcf9-6769-4317-8591-7c3f8828165d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:56.356604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 493/994] 10.222.0.23 () {56 vars in 1122 bytes} [Sat Sep 10 20:47:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:56.362782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:56.471207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b409ca-6349-4290-ae9f-4ac006a06b4c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:56.508331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d38a5-14a8-4b09-b199-35e3c584ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:56.508938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d38a5-14a8-4b09-b199-35e3c584ca40 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:56.509218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d38a5-14a8-4b09-b199-35e3c584ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:56.509584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d38a5-14a8-4b09-b199-35e3c584ca40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:56.510262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d38a5-14a8-4b09-b199-35e3c584ca40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:56.511617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d38a5-14a8-4b09-b199-35e3c584ca40 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:56.529139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 502/995] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:47:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:56.571352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ebb7-41fa-47dc-9790-32d8582fd72e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:56.571688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ebb7-41fa-47dc-9790-32d8582fd72e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:56.571930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ebb7-41fa-47dc-9790-32d8582fd72e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:56.573561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 494/996] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:56.579916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:56.580220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:56.580551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:56.600790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:56.602035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:56.656315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:56.695570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 503/997] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:56 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:56.962988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3ebb7-41fa-47dc-9790-32d8582fd72e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:57.070793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3ebb7-41fa-47dc-9790-32d8582fd72e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:57.104521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:57.105162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.105446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.105745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.106329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:57.107761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:57.116510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:57.117747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:47:57.118813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:57.121781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 495/998] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:47:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:57.124188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:57.227628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cecd9-84da-4941-879b-b08786b36eee None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:57.263649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:57.264319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.264603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.264873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.265841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:57.266143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:57.267562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68763d5d-afb8-45e3-b986-d1766d0b4636 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:57.277006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 504/999] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:57.278319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:57.344556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:57.388741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d077d-e0f2-4049-bf5b-9ae0855622f7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:57.423808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:57.424434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.424719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.424997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.425662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:57.425972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:57.427419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:57.429023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edb5cf-b831-442b-bc86-4a08ca212a39 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.429313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edb5cf-b831-442b-bc86-4a08ca212a39 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.429625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edb5cf-b831-442b-bc86-4a08ca212a39 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.431163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 505/1000] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:57.434234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 496/1001] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:47:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:57.438108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.438393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.438667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.440902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:57.462381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:57.463565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:57.494397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:57.534793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 506/1002] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:47:57.548804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4176b-6ebd-48a9-a839-e814e93f1936 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:57.573016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:57.585664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:57.586267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.586545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.586816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.587506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] RBAC: Adding query filter params (interface=public, service_id=cdf5a46c860346ad8c84753b7a48e94a, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:57.587812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:57.589150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:57.594415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 497/1003] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:47:57 2022] GET /identity/v3/endpoints?service_id=cdf5a46c860346ad8c84753b7a48e94a&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:47:57.599898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:57.700968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed94068-bcf8-4ba5-9dc9-166ffb78f0dc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:57.734647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:57.735252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.735529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.735798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.736393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:57.737773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:57.744118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 507/1004] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:47:57 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:47:57.808650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:57.924226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6e1f7-037c-408e-97b1-2ca7308fd60f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:57.961809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:57.962770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:57.963267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:57.963721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] PATH_INFO: `/v3/services/image` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:57.964531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:57.966117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:47:57.975140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:47:57.976128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:47:57.977069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:47:57.979418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 498/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Sep 10 20:47:57 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:47:57.985094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:58.017259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:58.087603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0576-5541-4fa6-8351-48bbbdd6702e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:58.121406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:58.122074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:58.122773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:58.122773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:58.123222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] RBAC: Adding query filter params (name=image) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:58.123521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:58.124872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c8d40-2503-4a32-921b-00c4f69d7507 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:58.129892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 508/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:58.135301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:58.191268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:58.259038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d0658-1452-4ad3-a1a8-3b5bcd2ec929 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:58.295678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:58.296303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:58.296591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:58.296870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:58.297519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] RBAC: Adding query filter params (type=image) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:47:58.297827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:58.299293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d072e-e4a9-455a-93d0-476603b37658 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:58.309494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 499/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:58.749353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012eca9-9e81-48c8-a62e-a05497f3323e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:58.749869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012eca9-9e81-48c8-a62e-a05497f3323e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:58.750154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012eca9-9e81-48c8-a62e-a05497f3323e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:58.751778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 509/1008] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:58.757020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:58.757379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:58.757770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:58.777316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:58.779371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:58.848190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 500/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:47:59.127476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012eca9-9e81-48c8-a62e-a05497f3323e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:59.228138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:59.238012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012eca9-9e81-48c8-a62e-a05497f3323e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:59.271986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384920d-9a94-4034-89ab-f99136fde077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:59.272617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384920d-9a94-4034-89ab-f99136fde077 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:59.272895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384920d-9a94-4034-89ab-f99136fde077 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:59.273168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384920d-9a94-4034-89ab-f99136fde077 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:59.274286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384920d-9a94-4034-89ab-f99136fde077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:59.275711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384920d-9a94-4034-89ab-f99136fde077 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:59.277507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 510/1010] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:47:59.283722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:47:59.328906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:59.383512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054c165-d394-413f-8f2f-fef39369df46 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:47:59.417319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de0e08-aeb7-4b98-84fb-6f29b80f5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:47:59.417963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de0e08-aeb7-4b98-84fb-6f29b80f5bd5 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:59.418246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de0e08-aeb7-4b98-84fb-6f29b80f5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:59.418519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de0e08-aeb7-4b98-84fb-6f29b80f5bd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:59.419865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de0e08-aeb7-4b98-84fb-6f29b80f5bd5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:47:59.421294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de0e08-aeb7-4b98-84fb-6f29b80f5bd5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:47:59.447963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 501/1011] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:47:59 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:47:59.954563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b49771-5600-4c5e-acd6-3ab1c95bc158 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:59.954883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b49771-5600-4c5e-acd6-3ab1c95bc158 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:59.955117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b49771-5600-4c5e-acd6-3ab1c95bc158 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:59.956863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 511/1012] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:59.961945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc793abf-c9d6-4873-a2fc-6a646c4589da None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:59.962235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc793abf-c9d6-4873-a2fc-6a646c4589da None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:59.962509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc793abf-c9d6-4873-a2fc-6a646c4589da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:59.980971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bc793abf-c9d6-4873-a2fc-6a646c4589da None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:59.982234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bc793abf-c9d6-4873-a2fc-6a646c4589da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:00.050247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 502/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:47:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:00.316216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b49771-5600-4c5e-acd6-3ab1c95bc158 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:00.428079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b49771-5600-4c5e-acd6-3ab1c95bc158 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:00.462276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:00.462907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.463242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.463516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:00.464086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:00.465439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:00.474541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:00.475531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:48:00.476394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:00.477996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 512/1014] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:48:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:00.481064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1938d27-30d9-46df-a246-74c776c33826 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.482053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1938d27-30d9-46df-a246-74c776c33826 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.482153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:00.482706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1938d27-30d9-46df-a246-74c776c33826 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:00.484362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 503/1015] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:00.489684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.489967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.490241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:00.508543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:00.509745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:00.514861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:00.532917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:00.584365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 504/1016] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:00.602476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043037f-f97c-4693-9843-4606e3ce8e99 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:00.637315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:00.638602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.638887 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.639164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:00.639793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:00.640099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:00.641763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:00.651475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 513/1017] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:48:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:00.652355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:00.768123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6677734-9fe7-4707-aaa2-7d175d2d46a5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:00.801877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:00.802493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.802778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.803057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:00.803693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:00.804002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:00.805367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:00.815373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 505/1018] 10.222.0.23 () {56 vars in 1112 bytes} [Sat Sep 10 20:48:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:00.816569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:00.869438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:00.897132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:00.928325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:00.939249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed8a2c-b6eb-4338-b6a3-92a5d099bba4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:00.978945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892fa18-a31a-4c74-a080-c3ecdba46e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:00.979558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892fa18-a31a-4c74-a080-c3ecdba46e87 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.979843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892fa18-a31a-4c74-a080-c3ecdba46e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.980119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892fa18-a31a-4c74-a080-c3ecdba46e87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:00.980786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892fa18-a31a-4c74-a080-c3ecdba46e87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:00.982388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892fa18-a31a-4c74-a080-c3ecdba46e87 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:00.986980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb69745-099f-4ff2-be2b-324789a34651 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.009645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 514/1019] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:48:01.041309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0ebe8-c6e9-4318-b0d6-52e2e7bbd51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.042012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0ebe8-c6e9-4318-b0d6-52e2e7bbd51f None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.042312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0ebe8-c6e9-4318-b0d6-52e2e7bbd51f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.042575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0ebe8-c6e9-4318-b0d6-52e2e7bbd51f None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.043255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0ebe8-c6e9-4318-b0d6-52e2e7bbd51f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.045212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0ebe8-c6e9-4318-b0d6-52e2e7bbd51f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.063820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 506/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:48:00 2022] POST /identity/v3/services => generated 242 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:48:01.687290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82ddad-09a2-49c0-ab0e-3e6324aeb625 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.687934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82ddad-09a2-49c0-ab0e-3e6324aeb625 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.687934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82ddad-09a2-49c0-ab0e-3e6324aeb625 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.689211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 515/1021] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:01.694546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.694946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.695147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.714006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.715191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.793042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 507/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:02.071471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82ddad-09a2-49c0-ab0e-3e6324aeb625 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.177425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82ddad-09a2-49c0-ab0e-3e6324aeb625 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.212242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266c4a-96bc-4b40-874a-193bed4c4aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.212863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266c4a-96bc-4b40-874a-193bed4c4aff None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.213141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266c4a-96bc-4b40-874a-193bed4c4aff None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.213408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266c4a-96bc-4b40-874a-193bed4c4aff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.214540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e266c4a-96bc-4b40-874a-193bed4c4aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.217020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e266c4a-96bc-4b40-874a-193bed4c4aff None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.218820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 516/1023] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.224302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.226494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:02.331397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401170ba-0d07-496f-b42e-2ef1bd2e6930 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.368344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb1dcd-9c21-4c99-aba6-56e9ab1cb0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.368966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb1dcd-9c21-4c99-aba6-56e9ab1cb0f3 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.369245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb1dcd-9c21-4c99-aba6-56e9ab1cb0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.369544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb1dcd-9c21-4c99-aba6-56e9ab1cb0f3 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.370212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb1dcd-9c21-4c99-aba6-56e9ab1cb0f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.371726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb1dcd-9c21-4c99-aba6-56e9ab1cb0f3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.401645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 508/1024] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:03.275478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954e620-496a-4ba0-9a15-47235cc5d8be None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.275998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954e620-496a-4ba0-9a15-47235cc5d8be None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.276081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954e620-496a-4ba0-9a15-47235cc5d8be None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.277363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 517/1025] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:03.282806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.283208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.283396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.302206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.303401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:03.335239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48295e2a-bfa9-4481-8925-9a7558bf7404 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.335561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48295e2a-bfa9-4481-8925-9a7558bf7404 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.335799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48295e2a-bfa9-4481-8925-9a7558bf7404 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.337147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 518/1026] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:03.342731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.343095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.343441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.364347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.365581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:03.366207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.376205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.384753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 509/1027] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:03.437506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 519/1028] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:48:03.658561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.706046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.768431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16effb3-f777-4ae6-9a6b-f560cd3053bd None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.806900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.807539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.807820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.808095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] PATH_INFO: `/v3/services/image` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.808686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.811107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.816048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4f70-ea30-4aca-a733-c0f3836d6d53 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:03.819959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:03.820959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:48:03.821864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:03.827138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 510/1029] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:03.828215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.855401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.856694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.857189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.857569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.868104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:03.873408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:03.874375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:03.875230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 520/1030] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:03.883729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.887794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.888653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.943369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6354561-22d6-4a8b-b62d-9cfb89f20a25 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.982706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.983344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.983637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.983916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.984640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] RBAC: Adding query filter params (name=image) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:03.984946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.986343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.991109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 511/1031] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:03.997449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.004207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a4d93-76b0-478b-9943-8d7dd7d88c67 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.041649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.042264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.042786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.042786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.043446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:04.044259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.045797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.054598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 521/1032] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:04.060257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.112451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27df22-0d54-4bd6-aaeb-c923943a7667 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.128253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.128914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.149568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.150181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.150487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.150743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.151372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] RBAC: Adding query filter params (type=image) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:04.151675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.153092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.158488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 512/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:04.164471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.173338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b743a0-208e-4e12-8186-efc5aae7788a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.209430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.210325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.210608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.211040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.221927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.223362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:04.237865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:04.238943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:04.239836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:04.239836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:04.239836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:04.239836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 522/1034] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:04.244533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.274246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41480a-29c4-40d5-9d6c-a508f980cc20 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.308467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.309104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.309413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.309726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.310583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] RBAC: Adding query filter params (interface=public, service_id=3b49344b32d34766b5fde91592d0d81e, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:04.310891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.312241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.324215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 513/1035] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/endpoints?service_id=3b49344b32d34766b5fde91592d0d81e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:04.326191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.358332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5cc59-d2ef-4b9c-85ab-f53a6ebfb323 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.375610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.378731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.398026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.398663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.399023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.399292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.399921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:04.400218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.401593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.416125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 523/1036] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:04.421666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.435615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6256992-47cd-4ad3-9c3e-023049b1d2a8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.470860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.471405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.471690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.471969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.472599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.474016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.484079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 514/1037] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:04.540558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94661f2-2c52-415f-8d3f-1518ebe1c30c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.582514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.583165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.583446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.583803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.592828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.594372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c7ad70fc-71b4-47b6-aed1-db5be8dc411d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:48:04.607797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:04.608844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:04.610627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 524/1038] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:04.614843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.650808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5b9f2-22e0-41f0-8e05-1565e22887cc None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.651150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5b9f2-22e0-41f0-8e05-1565e22887cc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.651377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5b9f2-22e0-41f0-8e05-1565e22887cc None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.652741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 525/1039] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:04.659573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.659870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.660158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.679882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:04.681065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:04.717181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.734842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f2558-177a-44df-a921-a10aa554c135 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.748315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.758618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 526/1040] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:04.780666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.781284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.781606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.781888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.782613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:04.782925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.784376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.807118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 515/1041] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:04.808500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.920468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82769d9-9adf-4cbd-ad8e-d73a66d66ad1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.955936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.956547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.956817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.957154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.993703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.028011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.028626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.030096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.135693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4d70d-913e-48ce-ba20-6f8f6a68edc3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.154187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 527/1042] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:48:05.160019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.176695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.177345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.177677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.177956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.189304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:05.194320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:05.195454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:05.196311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 516/1043] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:05.201327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.232566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.233222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.273835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabaccd-7212-4faa-a8f4-14aabfd5332d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.311618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.312356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.312633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.313013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.315872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473b376-dd2c-4908-b224-f9bcc63494f4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.323330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.324975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:05.341080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:05.342249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:05.343370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:05.343370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:05.343370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:05.343370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 528/1044] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:05.347988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.353025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.353667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.353949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.354281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.354947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:05.355251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.357315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.366125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 517/1045] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:05.371723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.459294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb13aa-37f3-429d-8302-37f146cf6634 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.476446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.480721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.491531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2b324-6aa5-48ef-8a33-b267d8c1d21a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.494685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.495300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.495574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.495847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.496482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:05.497269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.498755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.510510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 529/1046] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:05.516682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.528716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.529287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.535575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.535575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] PATH_INFO: `/v3/users/4d2ce656f35e408fb4c03f202ef480a5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.560738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] RBAC: Authorizing `identity:get_user(user_id=4d2ce656f35e408fb4c03f202ef480a5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.563434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.571053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 518/1047] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/users/4d2ce656f35e408fb4c03f202ef480a5 => generated 285 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:05.572069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.627743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76f96-e6ca-4f67-9e9f-9bc5ed956cde None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.662298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.662930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.663208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.663477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.670995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.672421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.677070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158b332-b432-41eb-a708-410ecf482468 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:48:05.683464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:05.685193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:48:05.687016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:05.689945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 530/1048] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:05.690861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.718837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.719690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.720121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.720546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.734254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.739993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.791946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ebff-8258-4ece-8256-ebc84c1a0e94 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.799465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.801033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.826904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.827518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.827795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.828066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.828750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:05.829055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.830494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.842175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 531/1049] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:05.848015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.917525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 519/1050] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:48:05.923909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.959152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.962721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6a9fe-2a7c-4195-89fc-3d57872ed5c0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.974149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.999326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.999939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.000221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.000496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.001217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] RBAC: Adding query filter params (role.id=860aa35e370b4ccdb3d60a5b458123e7, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=acfbe8c6a73d4b4992faba36c9881012) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:06.001563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.003007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.008528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 532/1051] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/role_assignments?user.id=acfbe8c6a73d4b4992faba36c9881012&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=860aa35e370b4ccdb3d60a5b458123e7 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:06.033570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618ac9f-713e-4e6e-a712-8d5ace8745f4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.072413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa602d7-6d8c-4d6d-937c-9d6fdca61f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.073500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa602d7-6d8c-4d6d-937c-9d6fdca61f82 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.073931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa602d7-6d8c-4d6d-937c-9d6fdca61f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.074299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa602d7-6d8c-4d6d-937c-9d6fdca61f82 None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.075672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa602d7-6d8c-4d6d-937c-9d6fdca61f82 None admin] RBAC: Authorizing `identity:get_project(project_id=e675bcda6b144511868b834398480032)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.077303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa602d7-6d8c-4d6d-937c-9d6fdca61f82 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.079372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 520/1052] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/projects/e675bcda6b144511868b834398480032 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:06.085380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.198099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93968f13-5d73-4b44-ac2c-fa41bfa2956a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.232751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.233366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.233678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.233953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.234666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=e675bcda6b144511868b834398480032, user.id=4d2ce656f35e408fb4c03f202ef480a5) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:06.234975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.236388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5986c0b-711b-48ca-a394-9fad01f7671c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.242101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 533/1053] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:06 2022] GET /identity/v3/role_assignments?user.id=4d2ce656f35e408fb4c03f202ef480a5&scope.project.id=e675bcda6b144511868b834398480032&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:06.705785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3861196-36df-4d22-a69b-181b91f8952d None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.706342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3861196-36df-4d22-a69b-181b91f8952d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.706342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3861196-36df-4d22-a69b-181b91f8952d None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.707811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 521/1054] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:06.712960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.713369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.713596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.733973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.735149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.758717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:06.805506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 534/1055] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:07.072760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3861196-36df-4d22-a69b-181b91f8952d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.177551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3861196-36df-4d22-a69b-181b91f8952d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.192267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:07.213164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.213825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.214104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.214377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] PATH_INFO: `/v3/services/image` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.214955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.216879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:07.225485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:07.226526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:07.228035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 522/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:07.234700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.337236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f58d1-cecf-442c-bb27-e78a1a188700 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.370852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.371618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.372495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.372832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.373558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] RBAC: Adding query filter params (name=image) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:07.373926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.375346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.384852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 535/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:07.385990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.491100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a4a88-6d9c-4ec8-9dd9-a81112c73232 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.525729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.526336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.526613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.526886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.527510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] RBAC: Adding query filter params (type=image) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:07.527832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.529557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75ea1e-6f2f-41a0-aa11-3e81a6e426cc None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.534370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 523/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:07.540121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.565987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:07.646623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefabd00-d51d-41ef-b334-85e3834f4908 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.680169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271f3c1-8ed6-4393-83e3-150ed156d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.680778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f3c1-8ed6-4393-83e3-150ed156d28f None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.681062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f3c1-8ed6-4393-83e3-150ed156d28f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.681338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f3c1-8ed6-4393-83e3-150ed156d28f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.682042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271f3c1-8ed6-4393-83e3-150ed156d28f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.683400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271f3c1-8ed6-4393-83e3-150ed156d28f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.706000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 536/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:08.187809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c749b47-3b68-4369-838d-5ec4031d318e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.188303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c749b47-3b68-4369-838d-5ec4031d318e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.188303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c749b47-3b68-4369-838d-5ec4031d318e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.189698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 524/1060] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:08.194871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a58813-f91b-48f7-9f20-31cd67125294 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.195252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a58813-f91b-48f7-9f20-31cd67125294 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.195434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a58813-f91b-48f7-9f20-31cd67125294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.213519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-07a58813-f91b-48f7-9f20-31cd67125294 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:08.214714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-07a58813-f91b-48f7-9f20-31cd67125294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.290812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 537/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:08.416480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7714ba9-0553-432a-9400-095391e622c9 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.416814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7714ba9-0553-432a-9400-095391e622c9 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.417028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7714ba9-0553-432a-9400-095391e622c9 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.419073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 525/1062] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:08.424536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.424831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.425111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.432430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.445558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:08.446698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.517086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 538/1063] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:08.563195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7714ba9-0553-432a-9400-095391e622c9 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.610840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.674050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7714ba9-0553-432a-9400-095391e622c9 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.707605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.708179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.708461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.708847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.719563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:08.724314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:08.725234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:48:08.726387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:08.729760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 526/1064] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:08.732221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.789908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.838480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a812-7669-4ae8-a787-290cbf67c766 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.874095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.874708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.874991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.875268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.875926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:08.876232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:08.877630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:08.879025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.888801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 539/1065] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:08.892748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.913473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a9f90-ef27-4bb7-b2e9-e2d6a791ab4c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.915019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.953033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.953714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.954000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.954280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.965058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:08.970028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:08.970984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:08.971822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 527/1066] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:08.976896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.005977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e24df2-3e77-4937-8bf7-46f56d7acd96 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.042430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.042960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.043238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.043543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.054222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.055911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:09.070707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:09.071730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:09.073260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:09.073260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:09.073260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:09.073260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 540/1067] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:09.078177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.094852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635b13f-006a-4152-8a97-8cd61b9228ef None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.128954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.131836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.132498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.133051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.133182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.133882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:09.134192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.135566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.145128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 528/1068] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:09.150843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.178170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.195120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5883b5-8daf-4cc9-9463-a1037b7f20dd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.231478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.231737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.232016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.232288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.232919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:09.233218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.234626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.247799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 541/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:09.252759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.261413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e493e-bfd4-4a22-81b6-e2052e5f8ac9 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.299986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.300633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.300945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.301202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] PATH_INFO: `/v3/users/4d2ce656f35e408fb4c03f202ef480a5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.302372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] RBAC: Authorizing `identity:get_user(user_id=4d2ce656f35e408fb4c03f202ef480a5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.303809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.305617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 529/1070] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/users/4d2ce656f35e408fb4c03f202ef480a5 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:09.311194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.354079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b21aa-145f-42b6-a026-01f1384000f8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.364214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.392138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.392826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.393116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.393395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.400814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.402286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:48:09.414193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:09.415202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:09.416170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 542/1071] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:09.419034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.421873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.430579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eea08-1c02-4bcf-b642-192550946f0a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.472139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.472783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.473070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.473342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.474516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] RBAC: Authorizing `identity:get_project(project_id=e675bcda6b144511868b834398480032)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.476029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.482560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 530/1072] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/projects/e675bcda6b144511868b834398480032 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:09.483492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.534338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca6ac-4657-48df-94c9-58a58aa114ea None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.571654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.572272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.572554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.572952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.573707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:09.574019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.576248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.586103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4fdbb-ec2e-42f2-887e-6ea033ca3a4c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.599985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 543/1073] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:09.601348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.622947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.623613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.623899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.624175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032/users/4d2ce656f35e408fb4c03f202ef480a5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.626135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] RBAC: Authorizing `identity:create_grant(project_id=e675bcda6b144511868b834398480032, user_id=4d2ce656f35e408fb4c03f202ef480a5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.627578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.630652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.644145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 531/1074] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:48:09 2022] PUT /identity/v3/projects/e675bcda6b144511868b834398480032/users/4d2ce656f35e408fb4c03f202ef480a5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:09.722365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eafd0a-82d2-4190-a1f4-c03a41106500 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.761327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.761985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.762890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.763822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.775481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.776963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a3446b5d-4b3f-4b74-8932-24ca2e0023f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:09.795041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:09.796079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:09.797001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:09.797001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:09.797001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:09.797001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 544/1075] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:09.802672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.830652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.916022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a9e8-f458-41df-8dde-60b9244969de None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.918138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690dfe5-b17a-4b93-873c-b82c8003385d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.919587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690dfe5-b17a-4b93-873c-b82c8003385d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.919866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690dfe5-b17a-4b93-873c-b82c8003385d None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.921375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 545/1076] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:48:09.927143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.927644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.928112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.948106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:09.949745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:09.954219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.954839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.955119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.955393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.956030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:09.956330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.957728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.970618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 532/1077] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:09.976794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.030614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 546/1078] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:48:10.066579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.087413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e9aa5-775c-4b0d-95d9-c796688f0c8d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.121096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.121906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.122189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.122461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.128976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.130445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:48:10.142442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:10.143343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:10.149054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 533/1079] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:48:10.151007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.163001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.256164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89ebf0-af50-4faf-9093-84978142b54c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.291074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.291615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.291894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.292168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.292860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:10.293162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.294490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.295689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.309539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 547/1080] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:10.314634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.387757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.408541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4927-4467-4797-84cb-4e78f3492028 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.423884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9819ea7-af60-47ca-9510-c4dd082a42d8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.444723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.445334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.445644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.445920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.457160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.461273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.461994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.462272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.462538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/acfbe8c6a73d4b4992faba36c9881012/roles/860aa35e370b4ccdb3d60a5b458123e7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.464711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=acfbe8c6a73d4b4992faba36c9881012, role_id=860aa35e370b4ccdb3d60a5b458123e7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.466724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.480199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 548/1081] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:48:10 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/acfbe8c6a73d4b4992faba36c9881012/roles/860aa35e370b4ccdb3d60a5b458123e7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:10.533601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.535095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.646414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 534/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:48:10.651927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.757517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ca00-5951-45a7-9774-47d3c03d041d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.785194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.795243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.795882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.796158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.796431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.807128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.808577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a299cb43-c0d7-41a0-9943-86a0d943d572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:10.822905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:10.823894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:10.825681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:10.825681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:10.825681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:10.825681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 549/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:10.830471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.873395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.943269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2feb246-0967-4c56-b624-86a995a561ec None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.981541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.981978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.982261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.982539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.983186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:10.983490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.984854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84988d-88c4-47d2-8250-e71e275ff83c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.005594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 535/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:11.890787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ee33e-794e-4855-a0ba-2058da855fb1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.891300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ee33e-794e-4855-a0ba-2058da855fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.891385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ee33e-794e-4855-a0ba-2058da855fb1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.892661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 550/1085] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:11.898228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.898622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.898900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.917402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:11.919205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.984131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 536/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:12.257763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ee33e-794e-4855-a0ba-2058da855fb1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.361167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ee33e-794e-4855-a0ba-2058da855fb1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.395627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.396264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.396548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.396823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.407588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:12.412330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:12.413599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:12.414519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 551/1087] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:12.419717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.448669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.520025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caec0a-f0e8-44a6-adf5-ff2692558e8b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.554767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.555404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.555686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.555965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.556649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:12.556954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.558398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbbf0f-5920-4400-8fd7-37de21096514 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.568325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 537/1088] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:12.575475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.619595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.668767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0380-8a66-456c-b60a-c94209fdfd86 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.669056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0380-8a66-456c-b60a-c94209fdfd86 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.669332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0380-8a66-456c-b60a-c94209fdfd86 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.670794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 538/1089] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:48:12.675695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.675985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.676261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.678865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401be7-4fc5-4235-83a8-eed0b5de1583 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.694363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:12.695549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.713573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.714201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.714477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.714748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] PATH_INFO: `/v3/users/4d2ce656f35e408fb4c03f202ef480a5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.715967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] RBAC: Authorizing `identity:get_user(user_id=4d2ce656f35e408fb4c03f202ef480a5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.717392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.718976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 552/1090] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:12 2022] GET /identity/v3/users/4d2ce656f35e408fb4c03f202ef480a5 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:12.724684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.765515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 539/1091] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:48:12.821676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaffed1-fc42-4bde-9fe6-0e5303f8e116 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.830763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.855411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.856616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.856892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.857159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.925643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.927089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.026885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 553/1092] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:48:13.033312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.033454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.043150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.056591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.130816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41102fba-4d14-4755-86d1-2c9d26bf087c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.132831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a05b86-364c-4db2-abc5-0f36139bfc4f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.164472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.165094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.165369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.165678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.166744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] RBAC: Authorizing `identity:get_project(project_id=e675bcda6b144511868b834398480032)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.167144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.167773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.168051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.168732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.169830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.170463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 554/1093] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/projects/e675bcda6b144511868b834398480032 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:13.176373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.179303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:13.184657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:13.185622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:13.187534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 540/1094] 10.222.0.23 () {56 vars in 1120 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:13.191640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.254411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.274958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.279252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201d03b-27cc-4f9e-9e64-ec1364adeea1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.294934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ed6ce-58b1-4089-bff7-21f46cc24f83 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.312837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.313439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.313753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.314036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.314757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=e675bcda6b144511868b834398480032, user.id=4d2ce656f35e408fb4c03f202ef480a5) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.315071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.316498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.322132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 555/1095] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/role_assignments?user.id=4d2ce656f35e408fb4c03f202ef480a5&scope.project.id=e675bcda6b144511868b834398480032&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:13.329147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.329813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.330160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.330439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.331118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.331424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.332836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.343547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 541/1096] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:13.349698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.461610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd03d-9bab-4b67-b529-a72309e3fced None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.497170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.498323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.498600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.498876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.507737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.510039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.511564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:13.527333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:13.528420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:13.529905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:13.529905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:13.529905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:13.529905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 556/1097] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:13.534390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.638910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e555f4-6dd1-4e93-9c46-4e4c19eba4f2 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.673839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.674450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.674730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.675000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.675633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.675934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.677290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.694067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 542/1098] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:13.695209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.795587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e618-a637-4908-8972-58e336e6725a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.828718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.829347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.829655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.829931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.837434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.838905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:48:13.850645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:13.851802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:13.852756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 557/1099] 10.222.0.23 () {56 vars in 1093 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:13.858337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.862681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.957424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5d85b-4228-428d-a997-e781ebfaedd8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.997155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.997910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.998196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.998474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.999209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.999513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.000938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.019204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 543/1100] 10.222.0.23 () {56 vars in 1133 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:14.024863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.080385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.127837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548d195-f464-4bf2-ae07-7454939aa815 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.160559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.161162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.161437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.161740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.226697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.228140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.284513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.331171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 558/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:14.337000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.398300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.437481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589f264-1831-4aa9-ac02-7e5248849214 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.471261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.471885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.472159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.472427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.482807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.484233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:14.498302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:14.499394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:48:14.500775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:14.500775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:48:14.500775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:14.500775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 544/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:14.505034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.605584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde681a-8f65-467d-9e31-1dc32366f6a5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.639060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.639661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.639938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.640472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.641097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:14.641394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.642770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.654519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 559/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:14.660053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.760314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013d320-70c0-41ad-b567-19fbc0f83e1f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.774290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.795509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.796141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.796426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.796701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.803194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.804635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:48:14.817663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:14.818560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:48:14.819465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:14.820211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 545/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:14.825753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.841541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.931980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478b14-4d8a-4b2c-b9b7-801a87e4df18 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.966785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.967389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.967671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.967945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.968622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:14.968926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.970369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec07045-3759-4039-9878-8967901df8a3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.983378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 560/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:14.989500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.086202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59ad2-4d14-4c28-b617-e40b5c8dab96 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:15.119161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:15.119765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.120043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.120314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.121024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] RBAC: Adding query filter params (role.id=860aa35e370b4ccdb3d60a5b458123e7, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=acfbe8c6a73d4b4992faba36c9881012) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:15.121372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:15.122822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013ac8-0584-48be-9f09-619fd9e3c4b0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:15.128413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 546/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/role_assignments?user.id=acfbe8c6a73d4b4992faba36c9881012&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=860aa35e370b4ccdb3d60a5b458123e7 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:15.525293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354639-8f01-4305-9724-461bd11f8350 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.525850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354639-8f01-4305-9724-461bd11f8350 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.525850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354639-8f01-4305-9724-461bd11f8350 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.527206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 561/1107] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:15.533398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.533870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.533975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.552611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:15.553996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:15.586811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:15.625129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 547/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:15.891187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354639-8f01-4305-9724-461bd11f8350 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.964514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:15.998865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354639-8f01-4305-9724-461bd11f8350 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.032271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba105ec-f3d8-4c59-b706-d13f81261563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.032906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba105ec-f3d8-4c59-b706-d13f81261563 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.033189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba105ec-f3d8-4c59-b706-d13f81261563 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.033511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba105ec-f3d8-4c59-b706-d13f81261563 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.034652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba105ec-f3d8-4c59-b706-d13f81261563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.036081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba105ec-f3d8-4c59-b706-d13f81261563 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.037683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 562/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:16.044064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.138368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f4f88-d886-4524-9f02-6bd5dadd020d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.171193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285dbcf-a87f-41e6-99e1-d77bb8928513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.171724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285dbcf-a87f-41e6-99e1-d77bb8928513 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.172000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285dbcf-a87f-41e6-99e1-d77bb8928513 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.172271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285dbcf-a87f-41e6-99e1-d77bb8928513 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.172907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285dbcf-a87f-41e6-99e1-d77bb8928513 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.174342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285dbcf-a87f-41e6-99e1-d77bb8928513 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.195901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 548/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:48:16 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:18.402195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db48d-6ca9-44a0-b28d-2bc83fcf9fda None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.402762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db48d-6ca9-44a0-b28d-2bc83fcf9fda None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.402868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db48d-6ca9-44a0-b28d-2bc83fcf9fda None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.404264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 563/1111] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:18.409905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280d430-7938-4f76-9bff-954225bef00d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.410275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280d430-7938-4f76-9bff-954225bef00d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.410458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280d430-7938-4f76-9bff-954225bef00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.428079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e280d430-7938-4f76-9bff-954225bef00d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:18.429248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e280d430-7938-4f76-9bff-954225bef00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:18.445836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:18.494854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 549/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:18.765716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db48d-6ca9-44a0-b28d-2bc83fcf9fda None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:18.865037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db48d-6ca9-44a0-b28d-2bc83fcf9fda None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:18.899124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:18.899754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.900041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.900306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] PATH_INFO: `/v3/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.901704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] RBAC: Authorizing `identity:get_role(role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:18.903654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:18.904988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 564/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:18 2022] GET /identity/v3/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:18.910827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280d430-7938-4f76-9bff-954225bef00d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.007528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280d430-7938-4f76-9bff-954225bef00d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.043482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.044169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.044455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.044734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] PATH_INFO: `/v3/users/21dbb7dcfd9d496db55b24086e890f5d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.057176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] RBAC: Authorizing `identity:get_user(user_id=21dbb7dcfd9d496db55b24086e890f5d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.059117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.060708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 550/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:18 2022] GET /identity/v3/users/21dbb7dcfd9d496db55b24086e890f5d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:19.066573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.078011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:19.164530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614cb52-9022-4087-b4e8-f3cb3dc95b60 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.197183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.197807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.198091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.198368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.263796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.265488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.274607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:19.361024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 565/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:19.367112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.381784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:19.467356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446c80e-939e-46e9-ab07-8348024eb494 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.500227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7e438-b819-4e22-b44a-a825ef6381d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.500851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7e438-b819-4e22-b44a-a825ef6381d1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.501134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7e438-b819-4e22-b44a-a825ef6381d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.501413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7e438-b819-4e22-b44a-a825ef6381d1 None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.502532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7e438-b819-4e22-b44a-a825ef6381d1 None admin] RBAC: Authorizing `identity:get_project(project_id=e675bcda6b144511868b834398480032)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.503949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7e438-b819-4e22-b44a-a825ef6381d1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.505577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 551/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:19 2022] GET /identity/v3/projects/e675bcda6b144511868b834398480032 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:19.512265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.608620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3574c0-e120-431d-96a7-4a2560826b8f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.617484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:19.642463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.643052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.643330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.643596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.644302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] RBAC: Adding query filter params (role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=e675bcda6b144511868b834398480032, user.id=21dbb7dcfd9d496db55b24086e890f5d) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:19.644606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.646054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499387-6e98-4d22-a3b1-7b9341e02b68 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.651172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 566/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:19 2022] GET /identity/v3/role_assignments?user.id=21dbb7dcfd9d496db55b24086e890f5d&scope.project.id=e675bcda6b144511868b834398480032&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:21.780285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767c2c9-021f-48aa-85fc-f92008a93149 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:21.780835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767c2c9-021f-48aa-85fc-f92008a93149 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:21.780835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767c2c9-021f-48aa-85fc-f92008a93149 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:21.782215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 552/1118] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:21.787427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:21.787869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:21.787970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:21.805866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:21.807042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:21.871786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 567/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:22.138415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767c2c9-021f-48aa-85fc-f92008a93149 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:22.219223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:22.239485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767c2c9-021f-48aa-85fc-f92008a93149 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:22.273752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:22.274374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:22.274655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:22.274929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] PATH_INFO: `/v3/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:22.276299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] RBAC: Authorizing `identity:get_role(role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:22.278766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:22.280071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 553/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:22 2022] GET /identity/v3/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:22.285870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:22.380924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c0621-5869-41ba-9aee-1ed39141a550 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:22.382433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:22.416354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:22.416923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:22.417197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:22.417507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] PATH_INFO: `/v3/users/21dbb7dcfd9d496db55b24086e890f5d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:22.419415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] RBAC: Authorizing `identity:get_user(user_id=21dbb7dcfd9d496db55b24086e890f5d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:22.420841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:22.422450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 568/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:22 2022] GET /identity/v3/users/21dbb7dcfd9d496db55b24086e890f5d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:22.428484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:22.527894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7692-905f-4317-a7c5-8405bffc655a None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:22.562300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68089c3-d25b-43bc-bc8e-26ecfb2fba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:22.562927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68089c3-d25b-43bc-bc8e-26ecfb2fba69 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:22.563209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68089c3-d25b-43bc-bc8e-26ecfb2fba69 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:22.563486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68089c3-d25b-43bc-bc8e-26ecfb2fba69 None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:22.564582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68089c3-d25b-43bc-bc8e-26ecfb2fba69 None admin] RBAC: Authorizing `identity:get_project(project_id=e675bcda6b144511868b834398480032)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:22.566059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68089c3-d25b-43bc-bc8e-26ecfb2fba69 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:22.567669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 554/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:22 2022] GET /identity/v3/projects/e675bcda6b144511868b834398480032 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:22.573566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:22.670876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b77708-ae83-4480-8995-6005f92b42a1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:22.704207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268313a3-6b31-415d-903a-c4fa25bcd839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:22.705128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268313a3-6b31-415d-903a-c4fa25bcd839 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:22.705655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268313a3-6b31-415d-903a-c4fa25bcd839 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:22.706201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268313a3-6b31-415d-903a-c4fa25bcd839 None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032/users/21dbb7dcfd9d496db55b24086e890f5d/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:22.710852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268313a3-6b31-415d-903a-c4fa25bcd839 None admin] RBAC: Authorizing `identity:create_grant(project_id=e675bcda6b144511868b834398480032, user_id=21dbb7dcfd9d496db55b24086e890f5d, role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:22.712301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268313a3-6b31-415d-903a-c4fa25bcd839 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:22.726094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 569/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:48:22 2022] PUT /identity/v3/projects/e675bcda6b144511868b834398480032/users/21dbb7dcfd9d496db55b24086e890f5d/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:24.834314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2a9b4-c6df-41af-8ceb-7b2c43d1efbf None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:24.834873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2a9b4-c6df-41af-8ceb-7b2c43d1efbf None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:24.834873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2a9b4-c6df-41af-8ceb-7b2c43d1efbf None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:24.836196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 555/1124] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:24.842009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:24.842428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:24.842564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:24.861651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:24.862834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:24.886324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:24.931955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 570/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:25.208118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2a9b4-c6df-41af-8ceb-7b2c43d1efbf None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:25.226429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:25.307771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2a9b4-c6df-41af-8ceb-7b2c43d1efbf None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:25.341313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:25.341960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.342246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.342523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] PATH_INFO: `/v3/roles/37d70251da4b43aaad59f5bf0c8377eb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.343905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] RBAC: Authorizing `identity:get_role(role_id=37d70251da4b43aaad59f5bf0c8377eb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:25.345270 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:25.346609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 556/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:25 2022] GET /identity/v3/roles/37d70251da4b43aaad59f5bf0c8377eb => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:25.352356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:25.449844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f49f-6c13-4cbd-9963-e42950e3666f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:25.482525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:25.483074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.483353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.483625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] PATH_INFO: `/v3/users/21dbb7dcfd9d496db55b24086e890f5d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.484725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] RBAC: Authorizing `identity:get_user(user_id=21dbb7dcfd9d496db55b24086e890f5d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:25.486175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:25.487723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 571/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:25 2022] GET /identity/v3/users/21dbb7dcfd9d496db55b24086e890f5d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:25.493363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:25.576653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:25.592219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb54c6-43ce-4459-b013-6a0154fef3ec None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:25.625002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:25.625627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.625913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.626189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.691098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:25.692536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:25.772138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:25.788288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 557/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:25.794729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:25.822282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:25.893307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259bf60-889c-4eb2-a7b1-e10a0f8a325b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:25.926867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96400acc-5df5-4037-8e42-0d8f0f7166a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:25.927489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96400acc-5df5-4037-8e42-0d8f0f7166a0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.927822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96400acc-5df5-4037-8e42-0d8f0f7166a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.928098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96400acc-5df5-4037-8e42-0d8f0f7166a0 None admin] PATH_INFO: `/v3/projects/e675bcda6b144511868b834398480032` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.929190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96400acc-5df5-4037-8e42-0d8f0f7166a0 None admin] RBAC: Authorizing `identity:get_project(project_id=e675bcda6b144511868b834398480032)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:25.930649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96400acc-5df5-4037-8e42-0d8f0f7166a0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:25.932297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 572/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:25 2022] GET /identity/v3/projects/e675bcda6b144511868b834398480032 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:25.939142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:26.033599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573239-af78-40f6-af7c-6e49a17e2c93 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:26.066228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:26.066814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:26.067092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:26.067361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:26.068068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] RBAC: Adding query filter params (role.id=37d70251da4b43aaad59f5bf0c8377eb, scope.project.id=e675bcda6b144511868b834398480032, user.id=21dbb7dcfd9d496db55b24086e890f5d) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:26.068371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:26.069810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d259619-6f01-48d6-aec8-44eabcc0daf6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:26.075382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 558/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:25 2022] GET /identity/v3/role_assignments?user.id=21dbb7dcfd9d496db55b24086e890f5d&scope.project.id=e675bcda6b144511868b834398480032&role.id=37d70251da4b43aaad59f5bf0c8377eb => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:28.236255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea6fd-9d71-478b-9d21-f300e2ff91a4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.236759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea6fd-9d71-478b-9d21-f300e2ff91a4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.236759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea6fd-9d71-478b-9d21-f300e2ff91a4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.238144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 573/1131] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:28.243426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.243844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.243986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.262676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:28.263861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.306345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:28.329931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 559/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:28.599296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ea6fd-9d71-478b-9d21-f300e2ff91a4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:28.693312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:28.700851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ea6fd-9d71-478b-9d21-f300e2ff91a4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:28.737412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612ae53-51be-43c0-8cd3-2fdf6c2f9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:28.738059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612ae53-51be-43c0-8cd3-2fdf6c2f9102 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.738332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612ae53-51be-43c0-8cd3-2fdf6c2f9102 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.738599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612ae53-51be-43c0-8cd3-2fdf6c2f9102 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.739678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612ae53-51be-43c0-8cd3-2fdf6c2f9102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:28.741096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612ae53-51be-43c0-8cd3-2fdf6c2f9102 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:28.742664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 574/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:28.748586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:28.847136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135add7-29dc-40d2-96aa-8171f37fd3f6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:28.880298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda663c-3d71-4df7-af13-f0a9e15ae889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:28.880849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda663c-3d71-4df7-af13-f0a9e15ae889 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.881130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda663c-3d71-4df7-af13-f0a9e15ae889 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.881402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda663c-3d71-4df7-af13-f0a9e15ae889 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.882095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda663c-3d71-4df7-af13-f0a9e15ae889 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:28.883530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda663c-3d71-4df7-af13-f0a9e15ae889 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:28.911416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 560/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:48:28 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:31.101411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32667b-2afb-4f92-97b5-10450a533ba1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.101971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32667b-2afb-4f92-97b5-10450a533ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.101971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32667b-2afb-4f92-97b5-10450a533ba1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.103353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 575/1135] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:31.109120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.109577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.109709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.128037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:31.129222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:31.152361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:31.195903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 561/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:31.462084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32667b-2afb-4f92-97b5-10450a533ba1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.561360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32667b-2afb-4f92-97b5-10450a533ba1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.594878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dff83-f2bf-4f1b-bdc1-682db675ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.595494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dff83-f2bf-4f1b-bdc1-682db675ca4b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.595769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dff83-f2bf-4f1b-bdc1-682db675ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.596036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dff83-f2bf-4f1b-bdc1-682db675ca4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.597162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dff83-f2bf-4f1b-bdc1-682db675ca4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.598616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dff83-f2bf-4f1b-bdc1-682db675ca4b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.600164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 576/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:48:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:31.606583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.706762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d246f-e3ce-4b90-90ad-9f6c4f58ccc9 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.749398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7dec-e40b-4b75-be0c-9f680b6b0b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.750237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7dec-e40b-4b75-be0c-9f680b6b0b0a None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.750653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7dec-e40b-4b75-be0c-9f680b6b0b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.751057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7dec-e40b-4b75-be0c-9f680b6b0b0a None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.752147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a7dec-e40b-4b75-be0c-9f680b6b0b0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.753604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a7dec-e40b-4b75-be0c-9f680b6b0b0a None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.777301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 562/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:33.946956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdbc74-e8bc-4025-89df-1d26c7ae4c63 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.947435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdbc74-e8bc-4025-89df-1d26c7ae4c63 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.947527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdbc74-e8bc-4025-89df-1d26c7ae4c63 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.948882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 577/1139] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:33.954221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.954629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.954784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.973240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:33.974467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:33.985745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:34.041072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 563/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:34.310875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdbc74-e8bc-4025-89df-1d26c7ae4c63 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.342208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:34.411545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdbc74-e8bc-4025-89df-1d26c7ae4c63 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.444694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.445313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.445619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.445896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.456504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:34.461222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:34.462372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:34.463239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 578/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:34.468326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.565641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b356-4637-4b5b-98e0-481425a81cf5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.598950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.599556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.599841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.600117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.600773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:34.601081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.602476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.610355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 564/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:34.616183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.709666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:34.719284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f847b20-024d-48c5-86e5-ab80ea1f9330 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.756970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.757622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.757904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.758180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] PATH_INFO: `/v3/users/5b454f85456b470285606676efa43a3e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.771954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] RBAC: Authorizing `identity:get_user(user_id=5b454f85456b470285606676efa43a3e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.773388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.775010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 579/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/users/5b454f85456b470285606676efa43a3e => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:34.780914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.792017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:34.879498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f7e2d-14c0-49c6-8f43-4c44de03f4ed None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.912515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.913124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.913401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.913700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.978348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.980390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.989532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:35.080899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 565/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:35.088453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:35.186857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b5cd6-e78c-4345-a338-a9117afb5eaa None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:35.219529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99065f62-7c69-4d84-a75c-5e436ce1c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:35.220150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99065f62-7c69-4d84-a75c-5e436ce1c30f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.220427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99065f62-7c69-4d84-a75c-5e436ce1c30f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.220697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99065f62-7c69-4d84-a75c-5e436ce1c30f None admin] PATH_INFO: `/v3/projects/f2f97600c0e94ebdbc958870950e7366` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.221796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99065f62-7c69-4d84-a75c-5e436ce1c30f None admin] RBAC: Authorizing `identity:get_project(project_id=f2f97600c0e94ebdbc958870950e7366)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:35.223209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99065f62-7c69-4d84-a75c-5e436ce1c30f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:35.224799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 580/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:35 2022] GET /identity/v3/projects/f2f97600c0e94ebdbc958870950e7366 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:35.231682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:35.326495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e0cd-0119-4cbe-93e5-f68f4a18fcf1 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:35.335218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:35.360823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:35.361427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.361739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.362007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.362720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=f2f97600c0e94ebdbc958870950e7366, user.id=5b454f85456b470285606676efa43a3e) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:35.363024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:35.364437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32094e3a-7ce4-4db0-8ea4-7b48c4d70f49 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:35.369625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 566/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:35 2022] GET /identity/v3/role_assignments?user.id=5b454f85456b470285606676efa43a3e&scope.project.id=f2f97600c0e94ebdbc958870950e7366&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:37.517297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d16b3-fc43-46fc-b807-81852186c79a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.517924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d16b3-fc43-46fc-b807-81852186c79a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.517924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d16b3-fc43-46fc-b807-81852186c79a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.519228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 581/1147] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:37.524266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.524708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.524807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.544492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:37.545699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:37.610821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 567/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:37.879512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d16b3-fc43-46fc-b807-81852186c79a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.899056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:37.980357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d16b3-fc43-46fc-b807-81852186c79a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.013178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.013830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.014112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.014386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.024473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:38.030022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:38.031227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:38.032500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 582/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:38.037129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.132418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ddc27-3155-40d6-9fab-83882497b594 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.133814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:38.167518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.168155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.168436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.168711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.169357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:38.169692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.171051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.178841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 568/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.185074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.261614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:38.283069 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfb4a1-8728-457d-8486-31ab239b0227 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.315675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.316286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.316564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.316836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] PATH_INFO: `/v3/users/5b454f85456b470285606676efa43a3e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.317957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] RBAC: Authorizing `identity:get_user(user_id=5b454f85456b470285606676efa43a3e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.319372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.320924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 583/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:38 2022] GET /identity/v3/users/5b454f85456b470285606676efa43a3e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.327450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.424168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a974-3a77-40f8-816e-ecbc51c05025 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.455767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffa292-71ff-4085-b7fd-226b8dbb51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.456393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffa292-71ff-4085-b7fd-226b8dbb51da None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.456677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffa292-71ff-4085-b7fd-226b8dbb51da None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.456955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffa292-71ff-4085-b7fd-226b8dbb51da None admin] PATH_INFO: `/v3/projects/f2f97600c0e94ebdbc958870950e7366` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.458075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffa292-71ff-4085-b7fd-226b8dbb51da None admin] RBAC: Authorizing `identity:get_project(project_id=f2f97600c0e94ebdbc958870950e7366)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.459498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffa292-71ff-4085-b7fd-226b8dbb51da None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.461099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 569/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:38 2022] GET /identity/v3/projects/f2f97600c0e94ebdbc958870950e7366 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.467583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.564923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6737bf-9ded-4ea7-a299-a2d92c7e437d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.598018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489f4b1-15b0-431a-9547-06d917d20e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.598660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489f4b1-15b0-431a-9547-06d917d20e73 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.598938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489f4b1-15b0-431a-9547-06d917d20e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.599210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489f4b1-15b0-431a-9547-06d917d20e73 None admin] PATH_INFO: `/v3/projects/f2f97600c0e94ebdbc958870950e7366/users/5b454f85456b470285606676efa43a3e/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.601066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489f4b1-15b0-431a-9547-06d917d20e73 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2f97600c0e94ebdbc958870950e7366, user_id=5b454f85456b470285606676efa43a3e, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.602520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489f4b1-15b0-431a-9547-06d917d20e73 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.615291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 584/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:48:38 2022] PUT /identity/v3/projects/f2f97600c0e94ebdbc958870950e7366/users/5b454f85456b470285606676efa43a3e/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:40.729411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfb42a-6e9d-43f2-92c8-5df889b62634 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:40.730166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfb42a-6e9d-43f2-92c8-5df889b62634 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:40.730503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfb42a-6e9d-43f2-92c8-5df889b62634 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:40.731899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 570/1154] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:40.737043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:40.737423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:40.737631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:40.751276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:40.756816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:40.758160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:40.824311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 585/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:41.093535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfb42a-6e9d-43f2-92c8-5df889b62634 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.125162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:41.193970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfb42a-6e9d-43f2-92c8-5df889b62634 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.226197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.226844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.227122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.227392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.237554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:41.242204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:41.243282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:41.244417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 571/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:41.249799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.345198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860fa3f-58fc-407e-93a1-79309e36b20e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.378564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.379161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.379440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.379710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.380363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:41.380668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.382060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.384189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:41.391295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 586/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:41.397110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.485237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:41.494772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d7491-b2f5-48bb-8c10-62aa410e58f7 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.528314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.528907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.529183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.529454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] PATH_INFO: `/v3/users/5b454f85456b470285606676efa43a3e` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.530578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] RBAC: Authorizing `identity:get_user(user_id=5b454f85456b470285606676efa43a3e)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.531999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.533572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 572/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/users/5b454f85456b470285606676efa43a3e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:41.539275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.638483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ae540-6e18-4f18-9028-5919d6fb6322 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.671379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.671980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.672262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.672537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.735037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:41.738970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.740420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.835451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 587/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:41.841598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.940983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec90574-7d32-4c2c-b528-bd6968b643d4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.973989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a58700-4858-4ae2-94f5-e726e86ebb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.974612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a58700-4858-4ae2-94f5-e726e86ebb2c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.974896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a58700-4858-4ae2-94f5-e726e86ebb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.975172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a58700-4858-4ae2-94f5-e726e86ebb2c None admin] PATH_INFO: `/v3/projects/f2f97600c0e94ebdbc958870950e7366` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.976278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a58700-4858-4ae2-94f5-e726e86ebb2c None admin] RBAC: Authorizing `identity:get_project(project_id=f2f97600c0e94ebdbc958870950e7366)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.977794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a58700-4858-4ae2-94f5-e726e86ebb2c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.980085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 573/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/projects/f2f97600c0e94ebdbc958870950e7366 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:41.986239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:42.078667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:42.082247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea3fb-8500-4188-9907-6667cbd44162 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:42.116333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:42.116927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:42.117209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:42.117504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:42.118222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=f2f97600c0e94ebdbc958870950e7366, user.id=5b454f85456b470285606676efa43a3e) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:42.118529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:42.119982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059a836-2f5c-4bf7-8fc7-79b9b91c3b63 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:42.125562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 588/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/role_assignments?user.id=5b454f85456b470285606676efa43a3e&scope.project.id=f2f97600c0e94ebdbc958870950e7366&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:44.277756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b51ee-7693-4e56-8e11-2f6d2f2287db None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.278270 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b51ee-7693-4e56-8e11-2f6d2f2287db None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.278388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b51ee-7693-4e56-8e11-2f6d2f2287db None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.279653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 574/1162] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:44.284752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.285127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.285300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.303672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:44.304848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.369165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 589/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:44.632946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b51ee-7693-4e56-8e11-2f6d2f2287db None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.662228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:44.734123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b51ee-7693-4e56-8e11-2f6d2f2287db None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.766879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.767480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.767764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.768042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.832713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.834173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.859871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:44.930496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 575/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:44.937236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.029400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:45.038469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cbe2e-b729-437b-9544-6b2963c557d1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.078893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.079525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.079805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.080078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.086549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.087981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-58d3bab4-eee2-4384-9ee1-98b0e3e0774a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:45.100450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:45.101591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:48:45.102553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 590/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:48:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:45.107833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.206594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371181b-8f51-4b68-8ceb-d8480986148e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.232309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:45.240129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.240731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.241015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.241293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.241960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:45.242266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.243625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b2ca6-d5b8-410e-aa68-5c9bf56fc417 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.251457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 576/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Sat Sep 10 20:48:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:47.374092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04402795-8b60-4345-b323-a5cdaa22af8a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:47.374597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04402795-8b60-4345-b323-a5cdaa22af8a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:47.374597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04402795-8b60-4345-b323-a5cdaa22af8a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:47.375899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 591/1167] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:47.381215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec6c90-ff81-4e2b-b649-0a7d3330f486 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:47.381698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec6c90-ff81-4e2b-b649-0a7d3330f486 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:47.381890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec6c90-ff81-4e2b-b649-0a7d3330f486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:47.400764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-11ec6c90-ff81-4e2b-b649-0a7d3330f486 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:47.401975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-11ec6c90-ff81-4e2b-b649-0a7d3330f486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:47.470205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 577/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:47.798422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04402795-8b60-4345-b323-a5cdaa22af8a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:47.915777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04402795-8b60-4345-b323-a5cdaa22af8a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:47.956977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c563380-b03e-4e21-a013-d9e6b0178ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:47.957618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c563380-b03e-4e21-a013-d9e6b0178ebe None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:47.957898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c563380-b03e-4e21-a013-d9e6b0178ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:47.958170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c563380-b03e-4e21-a013-d9e6b0178ebe None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:47.958832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c563380-b03e-4e21-a013-d9e6b0178ebe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:47.960386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c563380-b03e-4e21-a013-d9e6b0178ebe None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:47.977395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 592/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Sat Sep 10 20:48:47 2022] POST /identity/v3/domains => generated 267 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:50.141395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3828d45-393d-413a-9d49-43a0d8c9ca82 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.141975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3828d45-393d-413a-9d49-43a0d8c9ca82 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.141975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3828d45-393d-413a-9d49-43a0d8c9ca82 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.143355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 578/1170] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:50.148751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.149150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.149255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.168597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:50.169807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:50.176940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:50.237586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 593/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:50.508557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3828d45-393d-413a-9d49-43a0d8c9ca82 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:50.608679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3828d45-393d-413a-9d49-43a0d8c9ca82 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:50.630567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:50.642745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd9a3-03f4-4223-9e9c-f917c36d864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:50.643369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd9a3-03f4-4223-9e9c-f917c36d864b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.643648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd9a3-03f4-4223-9e9c-f917c36d864b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.643921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd9a3-03f4-4223-9e9c-f917c36d864b None admin] PATH_INFO: `/v3/domains/e9bc6cc8a08841838e6f8afbbd9684ba` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.654554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd9a3-03f4-4223-9e9c-f917c36d864b None admin] RBAC: Authorizing `identity:get_domain(domain_id=e9bc6cc8a08841838e6f8afbbd9684ba)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:50.655989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd9a3-03f4-4223-9e9c-f917c36d864b None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:50.657574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 579/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Sat Sep 10 20:48:50 2022] GET /identity/v3/domains/e9bc6cc8a08841838e6f8afbbd9684ba => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:50.663383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:50.759854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29977a-6fab-4a27-b75b-518848709380 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:50.793515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5131cc-57db-414d-a72a-f9e4cadccc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:50.794117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5131cc-57db-414d-a72a-f9e4cadccc70 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.794395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5131cc-57db-414d-a72a-f9e4cadccc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.794663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5131cc-57db-414d-a72a-f9e4cadccc70 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.795299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5131cc-57db-414d-a72a-f9e4cadccc70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:50.796714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5131cc-57db-414d-a72a-f9e4cadccc70 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:50.810535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:50.823180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 594/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:48:50 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:52.992451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f3e00-04ee-40c8-af08-ed598bcc40ee None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:52.993003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f3e00-04ee-40c8-af08-ed598bcc40ee None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:52.993003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f3e00-04ee-40c8-af08-ed598bcc40ee None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:52.994408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 580/1174] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:52.999619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743360ea-701d-4a86-9273-5abf65649570 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:53.000207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743360ea-701d-4a86-9273-5abf65649570 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:53.000207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743360ea-701d-4a86-9273-5abf65649570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:53.018735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-743360ea-701d-4a86-9273-5abf65649570 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:53.019902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-743360ea-701d-4a86-9273-5abf65649570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:53.085121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 595/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:53.359397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f3e00-04ee-40c8-af08-ed598bcc40ee None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:53.457955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f3e00-04ee-40c8-af08-ed598bcc40ee None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:53.490706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00b1cb-0bad-4b01-9630-4810d39b91cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:53.491324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b1cb-0bad-4b01-9630-4810d39b91cc None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:53.491599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b1cb-0bad-4b01-9630-4810d39b91cc None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:53.491871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b1cb-0bad-4b01-9630-4810d39b91cc None admin] PATH_INFO: `/v3/domains/e9bc6cc8a08841838e6f8afbbd9684ba` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:53.492961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00b1cb-0bad-4b01-9630-4810d39b91cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=e9bc6cc8a08841838e6f8afbbd9684ba)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:53.494491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00b1cb-0bad-4b01-9630-4810d39b91cc None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:53.496033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 581/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Sat Sep 10 20:48:53 2022] GET /identity/v3/domains/e9bc6cc8a08841838e6f8afbbd9684ba => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:53.501679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743360ea-701d-4a86-9273-5abf65649570 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:53.599349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743360ea-701d-4a86-9273-5abf65649570 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:53.621739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:53.632882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c6331-9ee1-42e8-b223-15c70eb95a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:53.633506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c6331-9ee1-42e8-b223-15c70eb95a89 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:53.633791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c6331-9ee1-42e8-b223-15c70eb95a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:53.634065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c6331-9ee1-42e8-b223-15c70eb95a89 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:53.634714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c6331-9ee1-42e8-b223-15c70eb95a89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:53.636685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c6331-9ee1-42e8-b223-15c70eb95a89 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:53.657987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 596/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:48:53 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:55.843827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd930-715f-4cd8-bad4-c060e9c0830e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:55.844357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd930-715f-4cd8-bad4-c060e9c0830e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:55.844357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd930-715f-4cd8-bad4-c060e9c0830e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:55.845772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 582/1178] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:55.851189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:55.851562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:55.851816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:55.871037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:55.872222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:55.939323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 597/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:56.207217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd930-715f-4cd8-bad4-c060e9c0830e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.259684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.311731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd930-715f-4cd8-bad4-c060e9c0830e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.344594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.345214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.345520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.345799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.356428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:56.361188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:56.362298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:56.363336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 583/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:56.368378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.464691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2f6a1-8cf5-4d5d-84cc-a177fb78243e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.474825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.498487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.499085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.499363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.499636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.500287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:56.500588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.501995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.510441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 598/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:56.516469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.616470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c0bf6-8186-4db6-ae2d-9ed9b837847b None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.626979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.649742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.650413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.650961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.650961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] PATH_INFO: `/v3/users/efc809f5594c487d83a8e17b15cceb87` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.663422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] RBAC: Authorizing `identity:get_user(user_id=efc809f5594c487d83a8e17b15cceb87)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.664843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.667060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 584/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/users/efc809f5594c487d83a8e17b15cceb87 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:56.672905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.769618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935dc12-d541-4f7e-8174-b63412a1c28c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.802252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.802842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.803118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.803388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.844056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.871117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.872544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.970774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 599/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:48:56.976740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.078898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794890-b992-4533-9837-2ec1c632f3a0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.113210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45012c48-764c-4ac4-b3c7-9f4b26dd521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:57.113892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45012c48-764c-4ac4-b3c7-9f4b26dd521d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.114166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45012c48-764c-4ac4-b3c7-9f4b26dd521d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.114442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45012c48-764c-4ac4-b3c7-9f4b26dd521d None admin] PATH_INFO: `/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:57.115545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45012c48-764c-4ac4-b3c7-9f4b26dd521d None admin] RBAC: Authorizing `identity:get_project(project_id=082823ada0fd4a03aa7c23d51fc6bb3a)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:57.116973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45012c48-764c-4ac4-b3c7-9f4b26dd521d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:57.118652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 585/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:57.124762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.193422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:57.222929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539275c3-dcb4-4a45-b940-b117b5d454ac None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.255600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:57.256189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.256469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.256737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:57.257442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=082823ada0fd4a03aa7c23d51fc6bb3a, user.id=efc809f5594c487d83a8e17b15cceb87) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:57.257776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:57.259178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3093e-66bc-4a9b-ade0-82e125b2fd00 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:57.265074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 600/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:48:57 2022] GET /identity/v3/role_assignments?user.id=efc809f5594c487d83a8e17b15cceb87&scope.project.id=082823ada0fd4a03aa7c23d51fc6bb3a&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:59.386974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6b1e0-8264-435f-b0b9-69d2082cbb04 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:59.387506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6b1e0-8264-435f-b0b9-69d2082cbb04 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:59.387506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6b1e0-8264-435f-b0b9-69d2082cbb04 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:59.388879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 586/1186] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:59.394181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:59.394560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:59.394725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:59.412846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:59.414039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:59.477543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 601/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:48:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:48:59.746741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6b1e0-8264-435f-b0b9-69d2082cbb04 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:59.781734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:59.845416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6b1e0-8264-435f-b0b9-69d2082cbb04 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:59.878264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:59.878875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:59.879154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:59.879423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:59.891305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:48:59.895991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:48:59.896955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:48:59.898661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 587/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:48:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:48:59.903827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:59.967005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:00.001876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e076753-91fd-4a95-94ca-0906a13059cb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.035374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.035974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.036253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.036524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.037185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:00.037511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.038879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.047291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 602/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:00.053230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.144207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:00.150914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8b11-80f9-45fa-b085-bcefe75fca9d None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.183875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.184499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.184781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.185058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] PATH_INFO: `/v3/users/efc809f5594c487d83a8e17b15cceb87` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.186200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] RBAC: Authorizing `identity:get_user(user_id=efc809f5594c487d83a8e17b15cceb87)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.187626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.189182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 588/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:49:00 2022] GET /identity/v3/users/efc809f5594c487d83a8e17b15cceb87 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:00.194865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.292759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba43d08-2aee-4f76-a5a0-653526398ed6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.318084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:00.326620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68f6f3-6319-461d-9885-ad5af3a56291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.327249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68f6f3-6319-461d-9885-ad5af3a56291 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.327524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68f6f3-6319-461d-9885-ad5af3a56291 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.327792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68f6f3-6319-461d-9885-ad5af3a56291 None admin] PATH_INFO: `/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.328880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68f6f3-6319-461d-9885-ad5af3a56291 None admin] RBAC: Authorizing `identity:get_project(project_id=082823ada0fd4a03aa7c23d51fc6bb3a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.330326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68f6f3-6319-461d-9885-ad5af3a56291 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.331997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 603/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:49:00 2022] GET /identity/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:00.338623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.434919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaff4e-a923-4bbd-a677-2aff488a1421 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.468431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf697e-dbc1-4e09-899d-2309aba1ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.469076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf697e-dbc1-4e09-899d-2309aba1ed96 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.469357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf697e-dbc1-4e09-899d-2309aba1ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.469659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf697e-dbc1-4e09-899d-2309aba1ed96 None admin] PATH_INFO: `/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a/users/efc809f5594c487d83a8e17b15cceb87/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.471526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf697e-dbc1-4e09-899d-2309aba1ed96 None admin] RBAC: Authorizing `identity:create_grant(project_id=082823ada0fd4a03aa7c23d51fc6bb3a, user_id=efc809f5594c487d83a8e17b15cceb87, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.472950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf697e-dbc1-4e09-899d-2309aba1ed96 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.485373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 589/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:49:00 2022] PUT /identity/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a/users/efc809f5594c487d83a8e17b15cceb87/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:02.585624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad440942-46c0-4be9-99df-966e66b6497e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:02.586178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad440942-46c0-4be9-99df-966e66b6497e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:02.586178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad440942-46c0-4be9-99df-966e66b6497e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:02.587505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 604/1193] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:02.592821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:02.593222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:02.593375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:02.611894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:02.613073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:02.630429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:02.679399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 590/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:49:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:49:02.946242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad440942-46c0-4be9-99df-966e66b6497e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.044988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad440942-46c0-4be9-99df-966e66b6497e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.078566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.079176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.079451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.079719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.089579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:49:03.094267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:49:03.095183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:49:03.096409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 605/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:49:03.101403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.199033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaabee-1d3c-4984-83bf-f25ee252c719 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.231263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.231864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.232144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.232417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.233070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:03.233374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:03.234754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:03.242550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 591/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:03.248898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.304152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:03.346857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae3a59-bb70-4a6d-9295-05d5c2c66542 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.380331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.380942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.381216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.381506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] PATH_INFO: `/v3/users/efc809f5594c487d83a8e17b15cceb87` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.382593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=efc809f5594c487d83a8e17b15cceb87)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:03.384017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:03.386281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 606/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Sat Sep 10 20:49:03 2022] GET /identity/v3/users/efc809f5594c487d83a8e17b15cceb87 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:03.392333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.411449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:03.490970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52733b4c-bbff-422e-afec-7539c9def798 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.523125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.523719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.523999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.524272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.589389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:03.590843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:03.606889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:03.685466 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 592/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:49:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:49:03.691407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.787693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194adbe-3e22-46d6-aa34-b3bc58c10cf5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.810847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:03.821615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fb213-27f2-476f-bf08-f14737a035f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.822811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fb213-27f2-476f-bf08-f14737a035f7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.823084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fb213-27f2-476f-bf08-f14737a035f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.823354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fb213-27f2-476f-bf08-f14737a035f7 None admin] PATH_INFO: `/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.824446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fb213-27f2-476f-bf08-f14737a035f7 None admin] RBAC: Authorizing `identity:get_project(project_id=082823ada0fd4a03aa7c23d51fc6bb3a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:03.825900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fb213-27f2-476f-bf08-f14737a035f7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:03.827496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 607/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Sat Sep 10 20:49:03 2022] GET /identity/v3/projects/082823ada0fd4a03aa7c23d51fc6bb3a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:03.833686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.933029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb7c47-5b5b-4b51-ba50-f7edb9428800 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.948528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:03.966272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.966869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.967177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.967450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.968161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=082823ada0fd4a03aa7c23d51fc6bb3a, user.id=efc809f5594c487d83a8e17b15cceb87) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:03.968465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:03.969918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c598c8c-4203-4c5f-9dab-830124b4904d None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:03.975386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 593/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:49:03 2022] GET /identity/v3/role_assignments?user.id=efc809f5594c487d83a8e17b15cceb87&scope.project.id=082823ada0fd4a03aa7c23d51fc6bb3a&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:13.818060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:13.972886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=10 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:00.078515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5964c-0c1d-4c0d-a851-23daaeea16dd None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:00.079114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5964c-0c1d-4c0d-a851-23daaeea16dd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:00.079114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5964c-0c1d-4c0d-a851-23daaeea16dd None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:00.081092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 608/1201] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:00.087830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:00.088301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:00.088388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:00.110121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:00.111488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:00.208121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 594/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:00.508536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5964c-0c1d-4c0d-a851-23daaeea16dd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:00.649664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5964c-0c1d-4c0d-a851-23daaeea16dd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:00.696151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:00.696931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:00.697212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:00.697500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:00.709648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:00.711914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:00.729700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:00.733690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:00.734974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:00.734974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:00.734974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:00.734974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 609/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:00.737242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:00.873544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e9a26-6b3f-4403-b00f-eaa207cc583a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:00.919932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:00.934442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:00.935206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:00.935518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:00.935795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:00.936579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:00.936921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:00.938563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266f7ec-39ca-4035-b067-1ce1d057a8c3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:00.951515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 595/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:00.958044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:01.024357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:01.085548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85685186-7c7c-42e9-8695-ec05df7a6529 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:01.131010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9026d-0a19-4b8a-91c5-bdc806545a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:01.131544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9026d-0a19-4b8a-91c5-bdc806545a5f None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:01.131869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9026d-0a19-4b8a-91c5-bdc806545a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:01.132162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9026d-0a19-4b8a-91c5-bdc806545a5f None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:01.132806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9026d-0a19-4b8a-91c5-bdc806545a5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:01.134375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9026d-0a19-4b8a-91c5-bdc806545a5f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:01.175768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 610/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Sat Sep 10 20:50:00 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:03.593047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f606eaf-c81b-4f82-a320-133c6223224c None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:03.593623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f606eaf-c81b-4f82-a320-133c6223224c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:03.593623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f606eaf-c81b-4f82-a320-133c6223224c None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:03.596636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 596/1206] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:03.602099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:03.602828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:03.602941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:03.622733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:03.623930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:03.701982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 611/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:03.982028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f606eaf-c81b-4f82-a320-133c6223224c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:04.102135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f606eaf-c81b-4f82-a320-133c6223224c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:04.149419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:04.150488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:04.150844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:04.151137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:04.166677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:50:04.171020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:04.171976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:04.173012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 597/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:04.178753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:04.226413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:04.300016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e31fcb-98af-426d-8990-0aa95ea4b03f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:04.341122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:04.341850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:04.342135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:04.342450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:04.343183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:04.343496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:04.344964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:04.359738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 612/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:04.360990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:04.423984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:04.486286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6f16a-9c85-446e-8f59-ef30a90ce2a0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:04.531314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:04.532082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:04.532463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:04.532780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:04.547705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:04.549184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:04.567859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:04.575428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:04.576715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:04.576715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:04.576715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:04.576715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 598/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:50:04.577019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:04.699265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:04.704401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd601a4-c68c-4569-b0af-7678ac5a8cd3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:04.744619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:04.745245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:04.745578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:04.745946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:04.746801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:04.747107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:04.748692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:04.764090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 613/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:04.769784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:04.891821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee581e78-0b5b-4a4c-a482-6fadcc4c9db3 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:04.899704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:04.932250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:04.932983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:04.933275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:04.933536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:04.942055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:04.943698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:50:04.957675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:04.958738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:04.959706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 599/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:50:04.965227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:05.082598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f146b7c-5221-4da5-8625-79efced6b4c2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:05.124484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:05.125151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:05.125501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:05.125780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:05.126504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:05.126809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:05.128328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:05.154638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 614/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Sep 10 20:50:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:05.155841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:05.273157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fa3c1-58ce-45ef-9960-57d9f32e7418 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:05.313290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:05.313938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:05.314221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:05.314496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:05.350086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:05.397638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:05.400629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:05.515714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 600/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:50:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:50:05.521286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:05.548454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:05.661286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe0b59-3cb7-4357-8b14-2123a02f8529 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:05.721217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:05.722004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:05.722354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:05.722640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:05.736822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:05.738567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:05.756870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:05.758400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:05.759443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:05.759443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:05.759443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:05.759443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 615/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:05.765053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:05.837940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:05.892164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d67729-3301-484c-96b3-60f392950f15 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:05.938720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:05.939359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:05.939647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:05.939927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:05.940626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:05.940929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:05.942550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:05.957132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 601/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:05.963106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:06.044611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:06.088311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05172f47-5b28-4a12-9d77-44a3bc4c49cd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:06.124979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:06.125871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:06.126187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:06.126615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:06.135752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:06.138136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:50:06.151634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:06.152644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:06.153700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 616/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:06.160815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:06.273224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1ebe-02d9-4d65-918c-5797576d85de None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:06.299925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:06.314554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:06.315374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:06.315796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:06.316389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:06.317254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:06.317684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:06.319115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2c23f-c8a2-45d7-bcd9-eada8cece1bf None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:06.333797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 602/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:50:06.341349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:06.456742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9614-6369-4945-bf36-d2ac0c3a72eb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:06.485908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:06.492806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:06.493703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:06.494112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:06.494423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:06.495378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=df16503b8f2e4d9bb3cff4275a39d994) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:06.495774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:06.497456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c3ab2-4a0a-41f6-ab42-5f87333b3be4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:06.503736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 617/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:50:06 2022] GET /identity/v3/role_assignments?user.id=df16503b8f2e4d9bb3cff4275a39d994&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:50:08.754968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b151fa7-25d6-49f8-a04d-29c95c3ebb57 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:08.755833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b151fa7-25d6-49f8-a04d-29c95c3ebb57 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:08.755833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b151fa7-25d6-49f8-a04d-29c95c3ebb57 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:08.756855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 603/1220] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:08.762755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:08.763208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:08.763313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:08.783311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:08.785181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:08.863176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 618/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:09.145626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b151fa7-25d6-49f8-a04d-29c95c3ebb57 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:09.260479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b151fa7-25d6-49f8-a04d-29c95c3ebb57 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:09.300657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:09.301296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:09.301613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:09.301886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:09.312587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:50:09.317330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:09.318690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:50:09.320127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:09.323433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 604/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:50:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:09.324549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:09.437976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367e985-98a7-4298-9e3b-4ed51881b5a8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:09.466403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:09.483107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:09.484072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:09.484556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:09.485015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:09.486093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:09.486615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:09.488589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:09.506327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 619/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:50:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:09.508068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:09.553798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:09.621998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608a75-7436-4dd3-8dcc-fc73a4f2152f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:09.660367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:09.661038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:09.661319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:09.661621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:09.672014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:09.673649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:09.689744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:09.690918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:09.692058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:09.692058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:09.692058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:09.692058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 605/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:09.697699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:09.807212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7649a68-e5c6-4245-a254-215b9439df85 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:09.845369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:09.846129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:09.846497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:09.846856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:09.847629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:09.848016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:09.849548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:09.865883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 620/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:09.872225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:09.976769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:09.984602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3d8fd-3732-4364-b982-c1d131721fb0 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:10.026339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:10.027195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:10.027463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:10.027837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:10.036773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:10.038433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:50:10.052092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:10.053316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:10.054553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 606/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:10.059923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:10.111497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:10.171125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b859ae5-660e-4f02-9e3e-85977ae85c3e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:10.209692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:10.210327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:10.210714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:10.211476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:10.212285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:10.212679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:10.214351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:10.234118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 621/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Sep 10 20:50:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:10.240268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:10.360305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47d423-e308-4a3f-9964-de985c6d2651 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:10.395913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:10.396642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:10.396922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:10.397271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:10.410709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:10.412468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:10.427979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:10.429069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:10.431037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:10.431037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:10.431037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:10.431037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 607/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:10.435481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:10.546953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1307612-6347-42e7-a851-2caad01b5a38 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:10.549276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:10.586396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:10.587168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:10.587461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:10.587762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:10.588471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:10.588819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:10.590428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:10.604469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 622/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:10.610754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:10.622559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:10.725149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccd783-a41b-4a30-ab2d-0ff89d863be1 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:10.764691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:10.765332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:10.765646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:10.766082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:10.773104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:10.774632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:50:10.787669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:10.789439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:10.790505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 608/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:50:10.795584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:10.906662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29029530-a6ef-4fd8-a6e1-f2836e87e235 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:10.944536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:10.945718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:10.946025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:10.946386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:10.947095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:10.947396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:10.948810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980f681-41fd-4952-8066-41f6b114f747 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:10.962957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 623/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:50:10.969264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:11.038704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:11.079046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a565c-18b7-4804-977d-7766e30a08eb None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:11.119217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dad067-9887-47e3-9c8f-951e6ed08c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:11.119887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dad067-9887-47e3-9c8f-951e6ed08c19 None admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:11.120173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dad067-9887-47e3-9c8f-951e6ed08c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:11.120495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dad067-9887-47e3-9c8f-951e6ed08c19 None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/df16503b8f2e4d9bb3cff4275a39d994/roles/faa2d892d00844b593fdac29781be6e0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:11.136377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dad067-9887-47e3-9c8f-951e6ed08c19 None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=df16503b8f2e4d9bb3cff4275a39d994, role_id=faa2d892d00844b593fdac29781be6e0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:11.138205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dad067-9887-47e3-9c8f-951e6ed08c19 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:11.151971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 609/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:50:10 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/df16503b8f2e4d9bb3cff4275a39d994/roles/faa2d892d00844b593fdac29781be6e0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:13.385635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a716-3d18-4c2c-b3de-d215611a578b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:13.386159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a716-3d18-4c2c-b3de-d215611a578b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:13.386159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a716-3d18-4c2c-b3de-d215611a578b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:13.387793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 624/1233] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:13.393390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:13.393870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:13.393962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:13.412689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:13.414011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:13.488756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 610/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:13.828266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a716-3d18-4c2c-b3de-d215611a578b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:13.857925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:13.950738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a716-3d18-4c2c-b3de-d215611a578b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:13.991954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:13.992872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:13.993152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:13.993425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.004422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:50:14.010274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:14.011504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:14.012766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 625/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Sat Sep 10 20:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:14.018472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:14.071070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:14.128144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2772228-46bd-4eb0-a47f-44686025d8de None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.166413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.167069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.167354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.167674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.168350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] RBAC: Adding query filter params (name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:14.168690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.170073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.178678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 611/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:14.185952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:14.282492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:14.302019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a73d9-ecc3-4aee-8556-9cff901f6ec2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.343223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.344032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.344318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.344655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.356484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.358001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:14.374596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:14.375871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:14.376988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:14.376988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:14.376988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:14.376988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 626/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:14.382540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:14.493850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c328c-0159-4f86-98f4-cb68ecf65887 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.503145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:14.531122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.531787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.532067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.532339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.533016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:14.533349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.534741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.547343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 612/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:14.553893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:14.681151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d05f-88fd-4c20-af1b-da7f82776b42 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.728310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.728979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.729305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.729610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.732711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:14.739049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.740798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:50:14.753947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:14.760389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:14.761697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 627/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:14.762722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:14.871541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626862b-ac92-45cb-9f34-e8b3768c37b4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.909262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.910153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.910430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.910705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.911434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:14.911740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.913164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.939429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 613/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:50:14.940396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:15.055370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20460161-26aa-44a5-83df-b7d39d91b119 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:15.094427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:15.095036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:15.095312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:15.095584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:15.165906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:15.172937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:15.174421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:15.284802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 628/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:50:15.293001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:15.327912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:15.407642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa523b12-7111-4a17-b376-777b1e273b1f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:15.445916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:15.446596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:15.446899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:15.447202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:15.460104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:15.461987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:15.480169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:15.481483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:15.482970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:15.482970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:15.482970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:15.482970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 614/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:15.487330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:15.594730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:15.601931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a7bc7-5564-4f0e-a416-3f2b75cfdd8c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:15.647200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:15.647973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:15.648292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:15.648596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:15.649356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:15.649726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:15.651259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:15.664538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 629/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:15.671020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:15.765078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:15.787884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360598d-87a9-40f3-856b-b5f84a639bcc None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:15.824597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:15.825226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:15.825542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:15.825813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:15.832613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:15.834166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:50:15.846333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:15.847330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:15.848366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 615/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:15.853488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:15.972010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283263f2-0ad1-464f-a9bb-92124b430427 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:16.008545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:16.009150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:16.009429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:16.009759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:16.010483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:16.010784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:16.012595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420076b8-dd0c-4792-96d8-4b43105a3b2c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:16.028221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 630/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:16.034880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:16.146862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554d75-ef19-41ae-a73b-c5d6c630d6c5 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:16.185761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:16.186458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:16.186748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:16.187017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:16.187767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] RBAC: Adding query filter params (role.id=faa2d892d00844b593fdac29781be6e0, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=df16503b8f2e4d9bb3cff4275a39d994) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:16.188130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:16.189695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea05f7-6aab-449d-afba-3df329d3c7c3 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:16.197294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 616/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:50:16 2022] GET /identity/v3/role_assignments?user.id=df16503b8f2e4d9bb3cff4275a39d994&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=faa2d892d00844b593fdac29781be6e0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:18.536024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17014ae-ecbd-491f-98ee-590c254d24df None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.536574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17014ae-ecbd-491f-98ee-590c254d24df None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.536686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17014ae-ecbd-491f-98ee-590c254d24df None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.538513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 631/1247] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:18.544259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.544775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.544922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.551561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:18.565241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:18.566669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:18.653378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 617/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:18.927804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17014ae-ecbd-491f-98ee-590c254d24df None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.948665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.041327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17014ae-ecbd-491f-98ee-590c254d24df None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.076543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.077212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.077558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.077834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.088714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:50:19.093786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:19.094971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:19.095829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 632/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:19.101696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.206803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a1716-fff2-496d-9395-aaaab11991dd None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.238490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.244942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.245624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.245898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.246175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.246899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:19.247203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.248678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.258000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 618/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:19.264897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.347197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.371515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60627b4-5c61-4d3a-8a83-1c70d61204c0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.407926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.408614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.408893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.409160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.420747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.422269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:19.438823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:19.440016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:19.440952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:19.440952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:19.440952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:19.440952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 633/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:19.447561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.562515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58963ab2-08c1-4777-ab5b-a5c7eba0806c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.600309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.601555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.601841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.602127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.602791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:19.603097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.604538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.618233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 619/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:19.624771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.736391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb490286-f1f4-4eea-812c-05929503442d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.764982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.774160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.774814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.775090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.775365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] PATH_INFO: `/v3/users/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.783811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.785279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:50:19.798198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:19.805180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:19.806420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 634/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:19.808624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.868433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.920909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140d2cf-5195-4b84-b18d-a96c9b367d7e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.958186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.959040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.959322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.959599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.960460 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:19.960775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.962286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.981048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 620/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:19.987119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.091159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f76f0-bad2-40f1-b0b9-ecd61f61bfec None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.125495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.126034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.126314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.126588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.182177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:20.194851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.196417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.292951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 635/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:50:20.298754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.393333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d77a73-9b73-4120-9906-46c8f89c4173 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.406344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:20.427097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.427713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.427990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.428262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.437957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.439377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:20.453722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:20.454810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:20.455906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:20.455906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:20.455906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:20.455906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 621/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:20.461045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.546540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:20.558847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243991d-1dff-4ae3-91fb-49c3b2c88c60 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.594655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.595365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.595781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.596178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.596999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:20.597441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.598993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.612560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 636/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:20.619510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.721923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ea26e-6738-4106-8d02-92578d484c10 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.753681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.754293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.754572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.754843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.761868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.763552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:50:20.774998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:20.776066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:20.777135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 622/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:50:20.781962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.880187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba88394-7be4-484f-83f0-42b85305c76a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.914417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.915019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.915294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.915563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.916239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:20.916533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.917982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbcfa4-75bc-4dfd-8749-91f03173f2f5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.930677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 637/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:20.936556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.962435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:21.035352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7347ba-dad9-4bd8-9220-f2f8cc1ce646 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:21.067287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:21.067880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:21.068155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:21.068425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:21.069138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=df16503b8f2e4d9bb3cff4275a39d994) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:21.070075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:21.071556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c76f3-7cc2-41f0-8bbf-124d6e071eed None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:21.076682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 623/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/role_assignments?user.id=df16503b8f2e4d9bb3cff4275a39d994&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:23.224744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97768-e1bd-486c-88e4-58d7ab2c3ef7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.225260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97768-e1bd-486c-88e4-58d7ab2c3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.225260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97768-e1bd-486c-88e4-58d7ab2c3ef7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.226732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 638/1261] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:23.231806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.232192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.232360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.251030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:23.252190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:23.332919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 624/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:23.608991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97768-e1bd-486c-88e4-58d7ab2c3ef7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:23.627818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:23.723927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97768-e1bd-486c-88e4-58d7ab2c3ef7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:23.756297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:23.757962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.758250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.758522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.768607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:50:23.775186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:23.776302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:23.777346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 639/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:23.782489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:23.800450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:23.880809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7145ed-6499-488b-a1bd-9f6c45ce75b5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:23.915415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:23.916035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.916319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.916595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.917254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:23.917592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:23.918957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:23.931091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 625/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:23.936786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.002876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.036286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5cbd5-c342-4da3-83d7-2e3bc2fd63d1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.070401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.071017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.071298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.071574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.081322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.082793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:24.097130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:24.098311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:24.099397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:24.099397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:24.099397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:24.099397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 640/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:24.104096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.183478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.202506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070c55c-8f47-4bc6-b58c-5aac60a4f57e None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.235058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.235647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.235935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.236206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.236869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:24.237175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.238569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.250592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 626/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:24.256194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.354910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2215ad-5561-406f-b023-36a55028be8e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.374433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.389078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.389735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.390020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.390296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.398525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.399954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:50:24.410801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:24.411926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:24.414168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 641/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:24.418312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.514777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf450c-4cad-44c1-918d-a0005eb4de61 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.547402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.547998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.548278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.548552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.549269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:24.549615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.551036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.559436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.568304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 627/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:24.574483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.678380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304a98-c6a8-4a2b-984d-c5819b5df8f7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.711682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.712296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.712571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.712840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.722880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.724312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:24.738081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:24.739689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:24.741850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:24.741850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:24.741850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:24.744298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 642/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:24.745347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.840605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff5e2d-9c2a-472d-a829-6894dfb2b23f None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.872774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.873362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.873664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.873935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.874561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:24.874859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.876215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.887440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 628/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:24.893015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.924456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.997072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041c8b1-a5e6-4d91-85ca-f9f936fb26fe None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.029382 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.030020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.030293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.030559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.036849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.038908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:50:25.049844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:25.050879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:25.051867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 643/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:25.057290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.115117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:25.157235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7901219-6a7f-42a3-a60b-fb50920664b6 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.190410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.191006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.191285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.191554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.192230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:25.192528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.193962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e2649-f330-4698-8457-1f4bc3f7561e None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.205442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 629/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:25.211545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.301968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:25.311436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a621fe-e90c-4c58-af42-1b2fcf8cb90d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.344722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d328c5-3013-4433-9078-3b365c29325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.345374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d328c5-3013-4433-9078-3b365c29325f None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.345688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d328c5-3013-4433-9078-3b365c29325f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.345965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d328c5-3013-4433-9078-3b365c29325f None admin] PATH_INFO: `/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/df16503b8f2e4d9bb3cff4275a39d994/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.347906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d328c5-3013-4433-9078-3b365c29325f None admin] RBAC: Authorizing `identity:create_grant(project_id=abec2d4d8c8e414092907eb50b7158b2, user_id=df16503b8f2e4d9bb3cff4275a39d994, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.349333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d328c5-3013-4433-9078-3b365c29325f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.362406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 644/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Sat Sep 10 20:50:25 2022] PUT /identity/v3/projects/abec2d4d8c8e414092907eb50b7158b2/users/df16503b8f2e4d9bb3cff4275a39d994/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:27.456582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e20dc-f32f-4861-9298-58fb1f075813 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.457148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e20dc-f32f-4861-9298-58fb1f075813 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.457148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e20dc-f32f-4861-9298-58fb1f075813 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.458481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 630/1274] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:27.463836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.464234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.464392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.482930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:27.484143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:27.549065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 645/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:27.836190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e20dc-f32f-4861-9298-58fb1f075813 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.936242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e20dc-f32f-4861-9298-58fb1f075813 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.953031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:27.969831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.970454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.970732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.971005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.980861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:50:27.985611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:27.986685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:27.987878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 631/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Sat Sep 10 20:50:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:27.992663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.071596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:28.090949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da63c9-f9e1-493b-a748-4173a4bf31a9 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.124482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.125077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.125354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.125674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.126328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:28.126629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.127979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.136244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 646/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Sat Sep 10 20:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:28.142263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.237711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf86b4-9a39-4afe-ae14-61ff7e2bccbf None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.269971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.270585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.270863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.271135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.281306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.282763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:28.296789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:28.297957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:28.298870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:28.298870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:28.298870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:28.298870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 632/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:28.303805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.402144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbdd0d-2a76-450f-8cb5-e461a95bd43d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.434667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.435252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.435531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.435801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.436426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:28.436725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.438102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.449972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 647/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:28.455554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.485604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:28.552746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f7f6-9680-43b0-bfe8-938a540b7e53 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.586175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.586861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.587177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.587449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.594734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.596164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:50:28.606925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:28.608025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:50:28.609203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:28.612642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 633/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:28.613806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.618475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:28.714514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c348dca-4c9e-4ed6-b0dd-0de1a7461da0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.747986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.748580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.748859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.749132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.750046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:28.750357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.751773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.768881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 648/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:28.774898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.862650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:28.872594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fe0ae-c7c9-4b98-990e-cd9a82af25f8 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.907184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.907798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.908081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.908353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.973991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.975990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:29.061112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:29.071278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 634/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:50:29.077023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:29.136437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:29.174765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab442e-fd21-475d-8e64-a801c11e645d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:29.207062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:29.207680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:29.207962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:29.208237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:29.218632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:29.220077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:50:29.234423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:29.235522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:50:29.236640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:50:29.236640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:50:29.236640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:29.236640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 649/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:29.241971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:29.336343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff2c7-a99a-46f1-9a73-b5c858ffcf9c None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:29.368713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:29.369306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:29.369617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:29.369889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:29.370518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:29.370817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:29.372156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:29.383665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 635/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Sat Sep 10 20:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:29.389538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:29.488672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d2c3-f561-4e38-b0c4-ba95ac7cb9d9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:29.502195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:29.523251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:29.523867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:29.524150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:29.524425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:29.531032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:29.532474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:50:29.545475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:29.546529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Sep 10 20:50:29.547651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 650/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Sat Sep 10 20:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:29.552462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:29.604144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:29.653041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788a2cd-0521-41a4-ad5f-2f9b9fe625ab None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:29.685362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:29.685983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:29.686262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:29.686530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:29.687203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:29.687503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:29.688905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bf49a-1906-4362-b66b-8aa0d68bb874 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:29.700475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 636/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Sat Sep 10 20:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:29.706686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:29.804376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1082-6795-463e-a1e1-c9c116d3c931 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:29.836283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:29.836868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:29.837143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:29.837411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:29.838152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] RBAC: Adding query filter params (role.id=8ab3a31c282a48a1b027ee33b288be76, scope.project.id=abec2d4d8c8e414092907eb50b7158b2, user.id=df16503b8f2e4d9bb3cff4275a39d994) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:29.838455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:29.839865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c715-946f-4cd3-a5da-74dc8637917b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:29.845340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 651/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Sat Sep 10 20:50:29 2022] GET /identity/v3/role_assignments?user.id=df16503b8f2e4d9bb3cff4275a39d994&scope.project.id=abec2d4d8c8e414092907eb50b7158b2&role.id=8ab3a31c282a48a1b027ee33b288be76 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:31.996911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a8a1-345f-4e4a-993e-37684b33cfd4 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:31.997427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a8a1-345f-4e4a-993e-37684b33cfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:31.997427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a8a1-345f-4e4a-993e-37684b33cfd4 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:31.998833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 637/1288] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:32.004155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.004554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.004687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.022648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:32.023810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:32.052539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:32.089473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 652/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:50:32.356814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a8a1-345f-4e4a-993e-37684b33cfd4 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:32.454653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a8a1-345f-4e4a-993e-37684b33cfd4 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.465034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:32.487791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.488408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.488686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.488956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.489548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.491500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:32.499867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:32.501097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:32.502118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 638/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:50:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:32.507011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:32.613660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e23a2-3141-4c03-b317-483f5828b7a0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.654560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.655163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.655436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.655710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.656508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:32.656700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.658044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cafa6-685b-47e1-9e4c-ca669bd133b0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:32.664557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 653/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Sep 10 20:50:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:32.670363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:32.769568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b819c4-13b8-409d-9504-0d7fd2c9c7fe None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.801776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.802368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.802648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.802922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] PATH_INFO: `/v3/services` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.804143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:32.804443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.805828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc4f0-6278-4c91-a0c8-2318ff3b1253 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:32.810090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 639/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Sep 10 20:50:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:34.981046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3531f09-a67e-4dae-929b-464d37034c66 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.981551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3531f09-a67e-4dae-929b-464d37034c66 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.981659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3531f09-a67e-4dae-929b-464d37034c66 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.982967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 654/1293] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:34.988049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02124eb-e14c-42d1-a5e8-6651a9f2ea17 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.988467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02124eb-e14c-42d1-a5e8-6651a9f2ea17 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.988609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02124eb-e14c-42d1-a5e8-6651a9f2ea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:35.007443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d02124eb-e14c-42d1-a5e8-6651a9f2ea17 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:35.008615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d02124eb-e14c-42d1-a5e8-6651a9f2ea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:35.038055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:35.075490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 640/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:35.349262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3531f09-a67e-4dae-929b-464d37034c66 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:35.380923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:35.453331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3531f09-a67e-4dae-929b-464d37034c66 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:35.487187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ef76-e681-4233-a109-09e178a15484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:35.487786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ef76-e681-4233-a109-09e178a15484 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:35.488066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ef76-e681-4233-a109-09e178a15484 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:35.488344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ef76-e681-4233-a109-09e178a15484 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:35.488981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ef76-e681-4233-a109-09e178a15484 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:35.490435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ef76-e681-4233-a109-09e178a15484 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:35.505299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 655/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Sat Sep 10 20:50:35 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:37.658812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e60e-3b0d-48c7-9d11-9da3e5ec4f87 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:37.659367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e60e-3b0d-48c7-9d11-9da3e5ec4f87 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:37.659367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e60e-3b0d-48c7-9d11-9da3e5ec4f87 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:37.660695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 641/1296] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:37.665904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:37.666275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:37.666450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:37.685061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:37.686266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:37.759536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 656/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:38.026453 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610e60e-3b0d-48c7-9d11-9da3e5ec4f87 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:38.126942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610e60e-3b0d-48c7-9d11-9da3e5ec4f87 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:38.160582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:38.161199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:38.161496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:38.161773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:38.163296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:38.164792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:38.173254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:38.174637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:38.175818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 642/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:50:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:38.180521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:38.198535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:38.279909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcac051-5a4f-4297-8777-26aaf151f970 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:38.314057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:38.314651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:38.314930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:38.315202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:38.315823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:38.316125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:38.317519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:38.322724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 657/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Sep 10 20:50:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:38.328646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:38.363245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:38.429564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bde4d-e336-4c25-b752-1e533533f1f4 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:38.462564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:38.463170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:38.463456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:38.463733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:38.464431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] RBAC: Adding query filter params (interface=public, service_id=e6c55b912d554abfab966c49ac4f8752, region_id=RegionOne) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:38.464739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:38.466132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba204a7-82ee-4ca6-894d-4e2871c35395 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:38.470970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 643/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Sat Sep 10 20:50:38 2022] GET /identity/v3/endpoints?service_id=e6c55b912d554abfab966c49ac4f8752&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:38.476830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:38.556845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:38.575222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e69a32-d792-477d-bd69-2205cb9ca136 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:38.613449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f4564-c437-410c-a2bc-91c00f75abd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:38.614213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4564-c437-410c-a2bc-91c00f75abd0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:38.614501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4564-c437-410c-a2bc-91c00f75abd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:38.614833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4564-c437-410c-a2bc-91c00f75abd0 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:38.615588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f4564-c437-410c-a2bc-91c00f75abd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:38.617196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f4564-c437-410c-a2bc-91c00f75abd0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:38.625735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 658/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Sat Sep 10 20:50:38 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:40.776594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717b62-7f8f-40a5-ac7e-6a0b521bba92 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.777181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717b62-7f8f-40a5-ac7e-6a0b521bba92 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.777181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717b62-7f8f-40a5-ac7e-6a0b521bba92 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.778525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 644/1302] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:40.783726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.784111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.784283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.803320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:40.804647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:40.869591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 659/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:50:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:50:41.137430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717b62-7f8f-40a5-ac7e-6a0b521bba92 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.230320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:41.236879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717b62-7f8f-40a5-ac7e-6a0b521bba92 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.268791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.269407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.269711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.269980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.270545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.272717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:50:41.281124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:50:41.282390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 20:50:41.283351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 645/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Sat Sep 10 20:50:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:50:41.288835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.361707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:41.384074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904b43-7222-459b-9bd5-f38b11b8346c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.417903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.418437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.418714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.418985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.419598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:41.419897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.421249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c89cb5-d9da-4d0d-8f3d-6b1dd5404fa1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.426214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 660/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Sat Sep 10 20:50:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:41.432078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.529623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3caef-ffc6-475d-9a01-a719a8c80863 None admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.561761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd2a12-46cc-453e-b6e4-e2072597a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.562351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd2a12-46cc-453e-b6e4-e2072597a228 None admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.562633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd2a12-46cc-453e-b6e4-e2072597a228 None admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.562905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd2a12-46cc-453e-b6e4-e2072597a228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.563563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd2a12-46cc-453e-b6e4-e2072597a228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.564908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd2a12-46cc-453e-b6e4-e2072597a228 None admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.584404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 646/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:51.423256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=30 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:51.579486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:46.000148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febebb7d-2bb3-4db5-8539-162b2ac1ced7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:46.001116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febebb7d-2bb3-4db5-8539-162b2ac1ced7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:46.002307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febebb7d-2bb3-4db5-8539-162b2ac1ced7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:46.004452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 661/1307] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:51:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:46.011405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607d284-bdf5-454d-8144-3443fe7450ee None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:46.013885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607d284-bdf5-454d-8144-3443fe7450ee None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:46.014378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607d284-bdf5-454d-8144-3443fe7450ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:46.165659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2607d284-bdf5-454d-8144-3443fe7450ee None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:46.167236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2607d284-bdf5-454d-8144-3443fe7450ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:46.346136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 647/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:51:46.630124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4127c-5e82-43f2-a949-e4485aa197c4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:46.630698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4127c-5e82-43f2-a949-e4485aa197c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:46.631138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4127c-5e82-43f2-a949-e4485aa197c4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:46.633021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 662/1309] 10.222.0.23 () {54 vars in 965 bytes} [Sat Sep 10 20:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:46.639912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca622-2e98-445c-9799-8430d58bf2ce None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:46.640209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca622-2e98-445c-9799-8430d58bf2ce None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:46.640488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca622-2e98-445c-9799-8430d58bf2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:46.726952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-05fca622-2e98-445c-9799-8430d58bf2ce None None] MFA Rules not processed for user `934be918dc8c4fa59988e5bb22e56eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:46.728198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-05fca622-2e98-445c-9799-8430d58bf2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:46.764921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:46.833494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 648/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Sat Sep 10 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:51:46.843647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4127c-5e82-43f2-a949-e4485aa197c4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:46.845598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3ba4127c-5e82-43f2-a949-e4485aa197c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:46.984404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4127c-5e82-43f2-a949-e4485aa197c4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:47.064210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:47.064876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:47.065146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:47.065397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:47.067061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:47.125823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:47.192808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:47.194231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360c173-08b7-4d2a-9f86-f8f3dde69380 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:47.215245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 663/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:49.754111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf58dd1-08d5-433f-9126-bbbfabeb19fd None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:49.754600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf58dd1-08d5-433f-9126-bbbfabeb19fd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:49.754719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf58dd1-08d5-433f-9126-bbbfabeb19fd None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:49.756032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 649/1312] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:49.761603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671637-702e-4d48-9ef6-61de1e85bdc6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:49.761984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671637-702e-4d48-9ef6-61de1e85bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:49.762154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671637-702e-4d48-9ef6-61de1e85bdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:49.832745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-61671637-702e-4d48-9ef6-61de1e85bdc6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:49.833977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-61671637-702e-4d48-9ef6-61de1e85bdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:49.934471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 664/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:50.226660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf58dd1-08d5-433f-9126-bbbfabeb19fd None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:50.228268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2bf58dd1-08d5-433f-9126-bbbfabeb19fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:50.338719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf58dd1-08d5-433f-9126-bbbfabeb19fd None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:50.407839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:50.410404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:50.411035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:50.411483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:50.412019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:50.413849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:50.522048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:50.523430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab617ef-5bfa-48ba-8fd2-e35a8d142231 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:50.541041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 650/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:53.044828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d6e7-1548-4c41-8225-2bafba0701ac None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:53.045347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d6e7-1548-4c41-8225-2bafba0701ac None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:53.045482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d6e7-1548-4c41-8225-2bafba0701ac None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:53.046850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 665/1315] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:53.053351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434887e7-31f8-49ef-a2a5-f5826b09ec87 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:53.053801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434887e7-31f8-49ef-a2a5-f5826b09ec87 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:53.053941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434887e7-31f8-49ef-a2a5-f5826b09ec87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:53.118405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-434887e7-31f8-49ef-a2a5-f5826b09ec87 None None] MFA Rules not processed for user `db0aec645bb84ac0b38455ee8fbc12af`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:53.119595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-434887e7-31f8-49ef-a2a5-f5826b09ec87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:53.171533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:53.216269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 651/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:51:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:53.517527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1064a65-c49d-4c76-b280-fc5725d48c7f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:53.518194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1064a65-c49d-4c76-b280-fc5725d48c7f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:53.518475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1064a65-c49d-4c76-b280-fc5725d48c7f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:53.519869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 666/1317] 10.222.0.23 () {54 vars in 965 bytes} [Sat Sep 10 20:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:53.525646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfffa1-a19f-44f0-ad39-afcefa49f73c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:53.525935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfffa1-a19f-44f0-ad39-afcefa49f73c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:53.526213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfffa1-a19f-44f0-ad39-afcefa49f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:53.605640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e8cfffa1-a19f-44f0-ad39-afcefa49f73c None None] MFA Rules not processed for user `934be918dc8c4fa59988e5bb22e56eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:53.606848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e8cfffa1-a19f-44f0-ad39-afcefa49f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:53.683580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 652/1318] 10.222.0.23 () {58 vars in 1062 bytes} [Sat Sep 10 20:51:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:53.692614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1064a65-c49d-4c76-b280-fc5725d48c7f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:53.694192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f1064a65-c49d-4c76-b280-fc5725d48c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:53.703893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:53.797685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1064a65-c49d-4c76-b280-fc5725d48c7f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:53.869927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:53.870530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:53.870812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:53.871087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:53.872692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:53.937662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:53.986760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:53.988204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e050d-f1e4-42cf-8841-b651b3684292 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:54.006144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 667/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:57.085305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6bb95-0546-4f3a-86fb-39be0b99ff24 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:57.085878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6bb95-0546-4f3a-86fb-39be0b99ff24 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:57.085878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6bb95-0546-4f3a-86fb-39be0b99ff24 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:57.087220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 653/1320] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:57.092510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da90c35-a9c4-4d95-b11a-7c51fe3704e9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:57.092959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da90c35-a9c4-4d95-b11a-7c51fe3704e9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:57.093055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da90c35-a9c4-4d95-b11a-7c51fe3704e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:57.160032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6da90c35-a9c4-4d95-b11a-7c51fe3704e9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:57.161150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6da90c35-a9c4-4d95-b11a-7c51fe3704e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:57.256006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 668/1321] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:57.550071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6bb95-0546-4f3a-86fb-39be0b99ff24 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:57.551618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aef6bb95-0546-4f3a-86fb-39be0b99ff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:57.605706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:57.662179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6bb95-0546-4f3a-86fb-39be0b99ff24 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:57.736519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:57.737131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:57.737414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:57.737721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:57.739316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:57.834763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:57.849181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:57.850724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a601c25-f426-441c-b608-4a767997fa3d service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:57.869768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 654/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:00.679862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e389973-78b2-446b-85ce-c9904322025f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:00.680440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e389973-78b2-446b-85ce-c9904322025f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:00.680440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e389973-78b2-446b-85ce-c9904322025f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:00.681842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 669/1323] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:00.687211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c313-4132-4a71-816c-ac09b8ae9365 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:00.687652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c313-4132-4a71-816c-ac09b8ae9365 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:00.687887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3c313-4132-4a71-816c-ac09b8ae9365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:00.763146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7fe3c313-4132-4a71-816c-ac09b8ae9365 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:00.764422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7fe3c313-4132-4a71-816c-ac09b8ae9365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:00.875219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 655/1324] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:01.154743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e389973-78b2-446b-85ce-c9904322025f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.156364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e389973-78b2-446b-85ce-c9904322025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:01.176520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:01.260736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e389973-78b2-446b-85ce-c9904322025f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.334234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.334839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.335111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.335378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.336938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:01.410470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:01.448670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.450199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790396d9-89a7-425b-9cbb-5236fa9ec0e6 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.468718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 670/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:03.678911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22b59d-b60c-43a3-b370-385b3dfb4cb4 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:03.679503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22b59d-b60c-43a3-b370-385b3dfb4cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:03.679503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22b59d-b60c-43a3-b370-385b3dfb4cb4 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:03.681254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 656/1326] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:03.688492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ec6b-8443-4257-9a04-5d2741368841 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:03.689047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ec6b-8443-4257-9a04-5d2741368841 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:03.689161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ec6b-8443-4257-9a04-5d2741368841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:03.757699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3f83ec6b-8443-4257-9a04-5d2741368841 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:03.759170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3f83ec6b-8443-4257-9a04-5d2741368841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:03.861212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 671/1327] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:04.147294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22b59d-b60c-43a3-b370-385b3dfb4cb4 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:04.148867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9c22b59d-b60c-43a3-b370-385b3dfb4cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:04.210646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:04.256504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22b59d-b60c-43a3-b370-385b3dfb4cb4 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:04.330041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:04.330652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:04.330930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:04.331198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:04.332765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:04.440008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:04.442802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:04.444330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61cb21-abc2-4ac1-9f13-5d64db2f8070 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:04.463261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 657/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:06.708089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e70c-efe8-4e37-90ae-39d2054eb4af None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:06.708637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e70c-efe8-4e37-90ae-39d2054eb4af None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:06.708637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e70c-efe8-4e37-90ae-39d2054eb4af None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:06.710022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 672/1329] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:06.715332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190ea63-02e4-409c-b154-1ff48c4f9f53 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:06.715737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190ea63-02e4-409c-b154-1ff48c4f9f53 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:06.715914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190ea63-02e4-409c-b154-1ff48c4f9f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:06.782176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8190ea63-02e4-409c-b154-1ff48c4f9f53 None None] MFA Rules not processed for user `db0aec645bb84ac0b38455ee8fbc12af`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:06.783379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8190ea63-02e4-409c-b154-1ff48c4f9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:06.880312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 658/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:07.163676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e70c-efe8-4e37-90ae-39d2054eb4af None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:07.165205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-be41e70c-efe8-4e37-90ae-39d2054eb4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:07.198616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:07.277860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e70c-efe8-4e37-90ae-39d2054eb4af None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:07.354394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:07.355066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:07.355347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:07.355616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:07.357184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:07.440415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:07.473493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:07.474950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32f405-04a2-47d0-9226-652456a2aa73 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:07.493074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 673/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:09.879513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1fab9-b850-41d3-83ba-ab785dd4d81e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:09.880048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1fab9-b850-41d3-83ba-ab785dd4d81e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:09.880048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1fab9-b850-41d3-83ba-ab785dd4d81e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:09.881421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 659/1332] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:09.886924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9770-4442-4119-a627-5a9edf31c1a8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:09.887351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9770-4442-4119-a627-5a9edf31c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:09.887469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9770-4442-4119-a627-5a9edf31c1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:09.953780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bf8d9770-4442-4119-a627-5a9edf31c1a8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:09.955008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf8d9770-4442-4119-a627-5a9edf31c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.050129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 674/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:10.331293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1fab9-b850-41d3-83ba-ab785dd4d81e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.332821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c5c1fab9-b850-41d3-83ba-ab785dd4d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.404256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:10.438488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1fab9-b850-41d3-83ba-ab785dd4d81e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.510666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.511271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.511550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.511822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.513388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.619331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.620681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1215b6-3f55-4754-93dd-1a4f4e7b411a service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.630428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:10.638655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 660/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:12.899061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241af80-a424-4caf-b86f-1ddbb2daaa64 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:12.899556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241af80-a424-4caf-b86f-1ddbb2daaa64 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:12.899662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241af80-a424-4caf-b86f-1ddbb2daaa64 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:12.900951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 675/1335] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:12.906257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296073f2-ca48-4d0f-ad77-e77d10e5c917 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:12.906774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296073f2-ca48-4d0f-ad77-e77d10e5c917 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:12.906878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296073f2-ca48-4d0f-ad77-e77d10e5c917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:12.972941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-296073f2-ca48-4d0f-ad77-e77d10e5c917 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:12.974100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-296073f2-ca48-4d0f-ad77-e77d10e5c917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:13.079859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 661/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:13.374338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241af80-a424-4caf-b86f-1ddbb2daaa64 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:13.376583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8241af80-a424-4caf-b86f-1ddbb2daaa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:13.421307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:13.485416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241af80-a424-4caf-b86f-1ddbb2daaa64 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:13.557878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:13.558475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:13.558749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:13.559014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:13.560581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:13.643650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:13.669394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:13.670865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe76dba-8670-4a0c-96d8-2d22f12b0be7 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:13.690216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 676/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:16.162152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145682f-5fd3-4d61-b653-dee53850643b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:16.162680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145682f-5fd3-4d61-b653-dee53850643b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:16.162680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145682f-5fd3-4d61-b653-dee53850643b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:16.163936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 662/1338] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:52:16.168958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e4e38-d984-4444-ac06-0596930a29f0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:16.169376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e4e38-d984-4444-ac06-0596930a29f0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:16.169522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e4e38-d984-4444-ac06-0596930a29f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:16.240400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a46e4e38-d984-4444-ac06-0596930a29f0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:16.241655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a46e4e38-d984-4444-ac06-0596930a29f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:16.337072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 677/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:16.617011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145682f-5fd3-4d61-b653-dee53850643b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:16.618619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7145682f-5fd3-4d61-b653-dee53850643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:16.693538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:16.722354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145682f-5fd3-4d61-b653-dee53850643b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:16.796007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:16.796652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:16.796930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:16.797205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:16.798816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:16.906817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:16.908268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eed511-c5bd-4365-95f9-354a1a5f8498 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:16.926548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 663/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:21.022048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c8938-fddf-4427-8505-2c15865f527b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:21.022633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c8938-fddf-4427-8505-2c15865f527b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:21.022633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c8938-fddf-4427-8505-2c15865f527b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:21.023927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 678/1341] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:21.029104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b02c-a614-40aa-a09f-ef361a1bcf35 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:21.029555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b02c-a614-40aa-a09f-ef361a1bcf35 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:21.029685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7b02c-a614-40aa-a09f-ef361a1bcf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:21.035889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:21.099451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-80c7b02c-a614-40aa-a09f-ef361a1bcf35 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:21.100556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-80c7b02c-a614-40aa-a09f-ef361a1bcf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:21.200473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 664/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:21.488141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c8938-fddf-4427-8505-2c15865f527b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:21.489804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7a9c8938-fddf-4427-8505-2c15865f527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:21.538760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:21.594785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c8938-fddf-4427-8505-2c15865f527b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:21.669850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:21.670456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:21.670732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:21.671001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:21.673146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:21.767866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:21.783944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:21.785374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f8124-fe15-43ba-b861-39d71ff96a19 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:21.804419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 679/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:24.604337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898ee32-227c-4b4d-9f2b-ed67451f3b4d None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:24.604969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898ee32-227c-4b4d-9f2b-ed67451f3b4d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:24.604969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898ee32-227c-4b4d-9f2b-ed67451f3b4d None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:24.606703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 665/1344] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:24.614090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f1c5-cf76-4dec-9ab3-9ec415fbc7a0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:24.614615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f1c5-cf76-4dec-9ab3-9ec415fbc7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:24.614615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f1c5-cf76-4dec-9ab3-9ec415fbc7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:24.681616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-26c8f1c5-cf76-4dec-9ab3-9ec415fbc7a0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:24.682832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-26c8f1c5-cf76-4dec-9ab3-9ec415fbc7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:24.778338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 680/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:25.058288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898ee32-227c-4b4d-9f2b-ed67451f3b4d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.059823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0898ee32-227c-4b4d-9f2b-ed67451f3b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.151699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:25.171834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898ee32-227c-4b4d-9f2b-ed67451f3b4d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.250895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.251508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.251784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.252053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.253658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.364045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.365522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32949c9-302f-4930-8d9b-b5c31ec549bf service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.385421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 666/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:29.575286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc63b-daf6-421f-adc2-bf63e23a8384 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.575799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc63b-daf6-421f-adc2-bf63e23a8384 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.575918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc63b-daf6-421f-adc2-bf63e23a8384 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.577688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 681/1347] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:29.582843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee40516-58f6-4498-8934-d12e3d8eb3bf None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.583259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee40516-58f6-4498-8934-d12e3d8eb3bf None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.583396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee40516-58f6-4498-8934-d12e3d8eb3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.602136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:29.657599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-aee40516-58f6-4498-8934-d12e3d8eb3bf None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:29.658822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aee40516-58f6-4498-8934-d12e3d8eb3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.767144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 667/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:30.054659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc63b-daf6-421f-adc2-bf63e23a8384 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.056195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-01adc63b-daf6-421f-adc2-bf63e23a8384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.114326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:30.161227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc63b-daf6-421f-adc2-bf63e23a8384 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.235600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.236125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.236400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.236669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.238280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.343409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:30.347286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.348721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c61e5a-770f-4cb7-ae94-6cc043481cae service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.367301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 682/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:35.261618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae400f3-2b04-4fd0-b8b3-e2fb72b37763 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.261618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae400f3-2b04-4fd0-b8b3-e2fb72b37763 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.261618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae400f3-2b04-4fd0-b8b3-e2fb72b37763 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.265103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 668/1350] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 20:52:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:35.271041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b814f-a28d-43da-b25c-123787ac82c4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.272915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b814f-a28d-43da-b25c-123787ac82c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.273195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b814f-a28d-43da-b25c-123787ac82c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.378985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bc4b814f-a28d-43da-b25c-123787ac82c4 None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:35.388106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bc4b814f-a28d-43da-b25c-123787ac82c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:35.544290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 683/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:39.738117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0955ef-21f2-4cb5-af72-ba3957c39781 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:39.738624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0955ef-21f2-4cb5-af72-ba3957c39781 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:39.738728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0955ef-21f2-4cb5-af72-ba3957c39781 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:39.740235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 669/1352] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 20:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:39.745312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0be3-4a21-4eb6-916b-b3be00842c5c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:39.746139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0be3-4a21-4eb6-916b-b3be00842c5c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:39.746482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0be3-4a21-4eb6-916b-b3be00842c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:39.765152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=59 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:39.804825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:39.830574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9f9a0be3-4a21-4eb6-916b-b3be00842c5c None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:39.831921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9f9a0be3-4a21-4eb6-916b-b3be00842c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:39.950344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 684/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:41.337984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa1783-d1d3-48db-a891-57c6846b2fc1 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:41.338511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa1783-d1d3-48db-a891-57c6846b2fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:41.338707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa1783-d1d3-48db-a891-57c6846b2fc1 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:41.340174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 670/1354] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 20:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:41.346039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254880e1-05c7-4116-85c2-f7852fa4563d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:41.346470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254880e1-05c7-4116-85c2-f7852fa4563d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:41.346738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254880e1-05c7-4116-85c2-f7852fa4563d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:41.425393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-254880e1-05c7-4116-85c2-f7852fa4563d None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:41.426631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-254880e1-05c7-4116-85c2-f7852fa4563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:41.470601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:41.536547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 685/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:45.171606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5f5e-42ce-442f-b8c9-1cd1a1f9b741 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.172152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5f5e-42ce-442f-b8c9-1cd1a1f9b741 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.172152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5f5e-42ce-442f-b8c9-1cd1a1f9b741 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.180638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83e10-7de5-4aac-a7b2-b5faf8f5e1b8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.181541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83e10-7de5-4aac-a7b2-b5faf8f5e1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.181986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83e10-7de5-4aac-a7b2-b5faf8f5e1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.185974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 671/1356] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:45 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:45.285972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-17f83e10-7de5-4aac-a7b2-b5faf8f5e1b8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:45.287171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-17f83e10-7de5-4aac-a7b2-b5faf8f5e1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:45.393839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 686/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:45.692598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8fbaf-c304-4cd5-82a0-5f15fd6c49c6 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.693296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8fbaf-c304-4cd5-82a0-5f15fd6c49c6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.693770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8fbaf-c304-4cd5-82a0-5f15fd6c49c6 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.695569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 672/1358] 10.222.0.23 () {54 vars in 962 bytes} [Sat Sep 10 20:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:45.705050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a64a2-9af9-4986-a679-27977536c153 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.705525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a64a2-9af9-4986-a679-27977536c153 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.706055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a64a2-9af9-4986-a679-27977536c153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.728720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:45.800526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b02a64a2-9af9-4986-a679-27977536c153 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:45.802182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b02a64a2-9af9-4986-a679-27977536c153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:45.938911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 687/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Sep 10 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:45.946883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8fbaf-c304-4cd5-82a0-5f15fd6c49c6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.948566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-92e8fbaf-c304-4cd5-82a0-5f15fd6c49c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:46.108328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8fbaf-c304-4cd5-82a0-5f15fd6c49c6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.189152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:46.246530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.247370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.248389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.248784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.250961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:46.355568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.358385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94842e20-c274-4c34-b772-ec1def80b484 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.377954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 673/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:48.795446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557769b3-213c-4b53-b03c-7e461b99a94e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:48.795446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557769b3-213c-4b53-b03c-7e461b99a94e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:48.796538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557769b3-213c-4b53-b03c-7e461b99a94e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:48.797271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 688/1361] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:48.802357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2ebd-60de-49da-b1bb-cdbeccdbdcb2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:48.802805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2ebd-60de-49da-b1bb-cdbeccdbdcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:48.802972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2ebd-60de-49da-b1bb-cdbeccdbdcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:48.877080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b1ff2ebd-60de-49da-b1bb-cdbeccdbdcb2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:48.878542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b1ff2ebd-60de-49da-b1bb-cdbeccdbdcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:48.924694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:48.987601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 674/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:49.274672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557769b3-213c-4b53-b03c-7e461b99a94e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:49.276253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-557769b3-213c-4b53-b03c-7e461b99a94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.356618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:49.460302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557769b3-213c-4b53-b03c-7e461b99a94e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:49.595706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:49.597035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:49.597035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:49.597351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:49.599059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.622217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:49.695628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:49.697222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d853-e48e-4dac-953d-1b1938ad83bb service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:49.717182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 689/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:52.259274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2796cb-7b4c-4b86-9d4c-c4d66a489924 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.259813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2796cb-7b4c-4b86-9d4c-c4d66a489924 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.260340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2796cb-7b4c-4b86-9d4c-c4d66a489924 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.262305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 675/1364] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:52.267494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e06288-d6f8-434d-8d2a-3c546ecbb2ad None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.267938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e06288-d6f8-434d-8d2a-3c546ecbb2ad None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.268023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e06288-d6f8-434d-8d2a-3c546ecbb2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.338981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b3e06288-d6f8-434d-8d2a-3c546ecbb2ad None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:52.340173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b3e06288-d6f8-434d-8d2a-3c546ecbb2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:52.449043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:52.452308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 690/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:52.737984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5e443-001d-4204-94b9-eb06c7ac87ff None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.738268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5e443-001d-4204-94b9-eb06c7ac87ff None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.738546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5e443-001d-4204-94b9-eb06c7ac87ff None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.739994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 676/1366] 10.222.0.23 () {54 vars in 962 bytes} [Sat Sep 10 20:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:52.745270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd9f5d-cbd5-4e6c-b880-5a1ad254f9b1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.745657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd9f5d-cbd5-4e6c-b880-5a1ad254f9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.746005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd9f5d-cbd5-4e6c-b880-5a1ad254f9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.830111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-42fd9f5d-cbd5-4e6c-b880-5a1ad254f9b1 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:52.831318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-42fd9f5d-cbd5-4e6c-b880-5a1ad254f9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:52.964451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 691/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Sep 10 20:52:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:52.974346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5e443-001d-4204-94b9-eb06c7ac87ff None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:52.975886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3bd5e443-001d-4204-94b9-eb06c7ac87ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.142067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5e443-001d-4204-94b9-eb06c7ac87ff None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:53.164947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:53.283352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:53.284715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.285514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.285798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.287445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.397554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:53.399178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daf146-0ec0-42e7-9076-f2a563a856b7 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:53.423115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 677/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:53.442727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b935b99-d822-4d4e-b5a0-f5cc5bbaf9fd None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.443270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b935b99-d822-4d4e-b5a0-f5cc5bbaf9fd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.443270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b935b99-d822-4d4e-b5a0-f5cc5bbaf9fd None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.444793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 692/1369] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:53.450738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9888177-4dcf-42e6-8708-9295cf6a497b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.451175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9888177-4dcf-42e6-8708-9295cf6a497b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.451463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9888177-4dcf-42e6-8708-9295cf6a497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.494426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:53.539795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c9888177-4dcf-42e6-8708-9295cf6a497b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:53.540970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c9888177-4dcf-42e6-8708-9295cf6a497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.714377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 678/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:54.133169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fee8b0-ae00-4d12-941c-b0e564681d9c None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.133169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fee8b0-ae00-4d12-941c-b0e564681d9c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.133169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fee8b0-ae00-4d12-941c-b0e564681d9c None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.133169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 693/1371] 10.222.0.23 () {54 vars in 964 bytes} [Sat Sep 10 20:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:52:54.139392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d851-9ac7-4748-a4aa-9b1e5f5265b5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.139392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d851-9ac7-4748-a4aa-9b1e5f5265b5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.139392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d851-9ac7-4748-a4aa-9b1e5f5265b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.247788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0836d851-9ac7-4748-a4aa-9b1e5f5265b5 None None] MFA Rules not processed for user `f07ba8c1958e4e2e82cdd4e490011593`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:54.248892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0836d851-9ac7-4748-a4aa-9b1e5f5265b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.329331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:54.370868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 679/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:54.379385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fee8b0-ae00-4d12-941c-b0e564681d9c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.380690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-69fee8b0-ae00-4d12-941c-b0e564681d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.439478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:54.507501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fee8b0-ae00-4d12-941c-b0e564681d9c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.595861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.596814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.596985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.597295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.598843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.706895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:54.723864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.725092 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09847c81-4475-4d9f-9f82-6426875a7bae service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.745792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 694/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:55.739253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a229-3e33-49a3-a2e9-5c336583701e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.739821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a229-3e33-49a3-a2e9-5c336583701e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.739821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a229-3e33-49a3-a2e9-5c336583701e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.741140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 680/1374] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:55.746514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7681c1d-a8ed-411d-ac70-159081a320d0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.746958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7681c1d-a8ed-411d-ac70-159081a320d0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.747757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7681c1d-a8ed-411d-ac70-159081a320d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.815631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b7681c1d-a8ed-411d-ac70-159081a320d0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:55.816853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b7681c1d-a8ed-411d-ac70-159081a320d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:55.915484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 695/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:56.201444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a229-3e33-49a3-a2e9-5c336583701e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:56.203111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-cf80a229-3e33-49a3-a2e9-5c336583701e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:56.368243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a229-3e33-49a3-a2e9-5c336583701e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:56.432512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:56.519748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:56.520379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:56.520655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:56.520924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:56.522900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:56.625057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:56.626746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dea15a-69dd-4491-a448-3927ecb18145 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:56.646046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 681/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:58.950923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51805a16-41d9-49fe-8fdf-6db713f904e1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:58.951481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51805a16-41d9-49fe-8fdf-6db713f904e1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:58.951481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51805a16-41d9-49fe-8fdf-6db713f904e1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:58.953244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 696/1377] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:58.958563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f4267-9c88-4254-a9fa-f5301ec0f0e8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:58.958965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f4267-9c88-4254-a9fa-f5301ec0f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:58.959152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f4267-9c88-4254-a9fa-f5301ec0f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:59.027159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-903f4267-9c88-4254-a9fa-f5301ec0f0e8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:59.028917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-903f4267-9c88-4254-a9fa-f5301ec0f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:59.036585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:59.141367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 682/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:59.432674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51805a16-41d9-49fe-8fdf-6db713f904e1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:59.434849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-51805a16-41d9-49fe-8fdf-6db713f904e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:59.499487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:59.595233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51805a16-41d9-49fe-8fdf-6db713f904e1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:59.715080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:59.716327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:59.716759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:59.717102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:59.718743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:59.738543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:59.814639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:59.816308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c504c3-85b2-49e4-b0ab-36a27f94c54f service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:59.835959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 697/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:02.065297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65721e-3f6e-4932-81f4-2836607bb111 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:02.065874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65721e-3f6e-4932-81f4-2836607bb111 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:02.066174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65721e-3f6e-4932-81f4-2836607bb111 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:02.067762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 683/1380] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:02.072939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e36a4-73a4-45ba-b6a3-1aa99b0b3ab9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:02.073415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e36a4-73a4-45ba-b6a3-1aa99b0b3ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:02.073521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e36a4-73a4-45ba-b6a3-1aa99b0b3ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:02.143659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d21e36a4-73a4-45ba-b6a3-1aa99b0b3ab9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:02.144877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d21e36a4-73a4-45ba-b6a3-1aa99b0b3ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:02.246623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:02.257515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 698/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:02.551302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65721e-3f6e-4932-81f4-2836607bb111 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:02.552827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-dd65721e-3f6e-4932-81f4-2836607bb111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:02.703275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:02.714310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65721e-3f6e-4932-81f4-2836607bb111 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:02.846166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:02.847023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:02.847380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:02.847728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:02.849908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:02.961565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:02.963114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292327-5d8c-4c36-aa8d-195963545a9a service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:02.973830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:02.982438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 684/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:05.310670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41abd1c-ea55-49b5-9441-180e5cec6555 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:05.311250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41abd1c-ea55-49b5-9441-180e5cec6555 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:05.311410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41abd1c-ea55-49b5-9441-180e5cec6555 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:05.313191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 699/1383] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:05.320765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbc170-e519-4c93-b817-f514b3a7ab6b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:05.321239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbc170-e519-4c93-b817-f514b3a7ab6b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:05.321342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbc170-e519-4c93-b817-f514b3a7ab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:05.390540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f0cbc170-e519-4c93-b817-f514b3a7ab6b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:05.391892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f0cbc170-e519-4c93-b817-f514b3a7ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:05.512778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 685/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:05.906834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41abd1c-ea55-49b5-9441-180e5cec6555 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:05.908406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d41abd1c-ea55-49b5-9441-180e5cec6555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:06.072025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41abd1c-ea55-49b5-9441-180e5cec6555 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:06.143992 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:06.193221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:06.193893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.194173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.194425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.196063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:06.291871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:06.293604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da414c-cd98-47a8-ab10-bc3e59eb22a3 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.312422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 700/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:08.559191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289fa65-dcdb-4487-8b89-32f038e89bae None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:08.559758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289fa65-dcdb-4487-8b89-32f038e89bae None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:08.559758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289fa65-dcdb-4487-8b89-32f038e89bae None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:08.561121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 686/1386] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:08.566564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734c5cd-1899-4a8c-b1fc-1c37bbdae013 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:08.566951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734c5cd-1899-4a8c-b1fc-1c37bbdae013 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:08.567112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734c5cd-1899-4a8c-b1fc-1c37bbdae013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:08.632209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9734c5cd-1899-4a8c-b1fc-1c37bbdae013 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:08.633413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9734c5cd-1899-4a8c-b1fc-1c37bbdae013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:08.666739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:08.733149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 701/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:09.022057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289fa65-dcdb-4487-8b89-32f038e89bae None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:09.023576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b289fa65-dcdb-4487-8b89-32f038e89bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:09.099015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:09.170653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289fa65-dcdb-4487-8b89-32f038e89bae None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:09.281802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:09.282411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:09.282687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:09.282956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:09.284525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:09.322587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:09.375668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:09.377252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc079fbf-3a1b-425a-88b5-f4ab37e8b302 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:09.395869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 687/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:10.180962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12139395-2517-42fb-b654-72e396c41c3a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.181529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12139395-2517-42fb-b654-72e396c41c3a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.181529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12139395-2517-42fb-b654-72e396c41c3a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.182902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 702/1389] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:10.188726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a17cf5-9946-421b-a69c-0fd90a338907 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.189167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a17cf5-9946-421b-a69c-0fd90a338907 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.189285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a17cf5-9946-421b-a69c-0fd90a338907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.257084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-90a17cf5-9946-421b-a69c-0fd90a338907 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:10.258331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-90a17cf5-9946-421b-a69c-0fd90a338907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.361009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 688/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:10.667361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712020-8eb1-42e2-b4ba-603f12d31d32 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.667688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712020-8eb1-42e2-b4ba-603f12d31d32 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.668385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712020-8eb1-42e2-b4ba-603f12d31d32 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.669847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 703/1391] 10.222.0.23 () {54 vars in 966 bytes} [Sat Sep 10 20:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:10.675498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7819-b367-4c89-a00c-ac6f27aa0f4f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.675792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7819-b367-4c89-a00c-ac6f27aa0f4f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.676073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7819-b367-4c89-a00c-ac6f27aa0f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.691906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:10.753302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-32cc7819-b367-4c89-a00c-ac6f27aa0f4f None None] MFA Rules not processed for user `ee9a524cfaa141409ff24c149b698be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:10.754551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-32cc7819-b367-4c89-a00c-ac6f27aa0f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.830635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 689/1392] 10.222.0.23 () {58 vars in 1063 bytes} [Sat Sep 10 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:10.839677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712020-8eb1-42e2-b4ba-603f12d31d32 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:10.841589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1b712020-8eb1-42e2-b4ba-603f12d31d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.947935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712020-8eb1-42e2-b4ba-603f12d31d32 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.007888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:11.024250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.024855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.025141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.025413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.027000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:11.151180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.154220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33d8d0-b395-4b3c-bb38-0f56536b8287 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.173824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 704/1393] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:11.579322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764ecd1-655b-4bda-82ae-a9135588f59f None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.579860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764ecd1-655b-4bda-82ae-a9135588f59f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.579860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764ecd1-655b-4bda-82ae-a9135588f59f None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.581214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 690/1394] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:11.586719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.587167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.587346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.657417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:11.658659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:11.705426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:11.771587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 705/1395] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:11.952054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bab00-c89c-45ae-a512-ff0e282d90ef None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.952344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bab00-c89c-45ae-a512-ff0e282d90ef None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.952622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bab00-c89c-45ae-a512-ff0e282d90ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.020929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f44bab00-c89c-45ae-a512-ff0e282d90ef None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:12.022178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f44bab00-c89c-45ae-a512-ff0e282d90ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.066414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:12.067857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.078187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:12.112511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 691/1396] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:12.160286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca2521-877d-4a65-9d85-e3cb5039976a None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.160579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca2521-877d-4a65-9d85-e3cb5039976a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.160859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca2521-877d-4a65-9d85-e3cb5039976a None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.162532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 692/1397] 10.222.0.23 () {54 vars in 943 bytes} [Sat Sep 10 20:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:53:12.168995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.169239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.169625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.198083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd89c-efb8-420f-96c4-d0e63589990d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:12.249641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] MFA Rules not processed for user `233de9dbbb7a48159a1b0acdcd9880b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:12.250927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.254737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:12.326295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:12.328000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.328000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.328000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.329130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.374863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 693/1398] 10.222.0.23 () {58 vars in 1040 bytes} [Sat Sep 10 20:53:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:12.384529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:12.395291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea0210-2318-43a9-b396-ff3e2250629a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:12.423947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:12.424444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:12.426118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4ba4d-4f98-4ec9-9e5f-6c3747232be4 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:12.445349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 706/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:53:12.511803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:12.512429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.512780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.513050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.515528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.610041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:12.611724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde2eb-cf0f-430b-ae25-77ec1dc38319 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:12.621453 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 694/1400] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 20:53:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 20:53:14.704591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1d0e0-ad09-4022-9cc0-08968ef4d415 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.705267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1d0e0-ad09-4022-9cc0-08968ef4d415 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.705267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1d0e0-ad09-4022-9cc0-08968ef4d415 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.706515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 707/1401] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:14.711786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303873f0-189e-4686-8620-eab43547dcdd None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.712223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303873f0-189e-4686-8620-eab43547dcdd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.712383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303873f0-189e-4686-8620-eab43547dcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.785582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-303873f0-189e-4686-8620-eab43547dcdd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:14.786921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-303873f0-189e-4686-8620-eab43547dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:14.794795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:14.912815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 695/1402] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:15.206818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1d0e0-ad09-4022-9cc0-08968ef4d415 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:15.208424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-53a1d0e0-ad09-4022-9cc0-08968ef4d415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:15.282602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:15.379020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1d0e0-ad09-4022-9cc0-08968ef4d415 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:15.510371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:15.511414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:15.511919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:15.512248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:15.514011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:15.545606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:15.620438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:15.622676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b777d-b11e-4641-8a4a-4cc82e6ad07d service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:15.642365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 708/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:17.862427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66821b38-ec37-48d6-a7da-cc94b0428893 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:17.862957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66821b38-ec37-48d6-a7da-cc94b0428893 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:17.862957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66821b38-ec37-48d6-a7da-cc94b0428893 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:17.864536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 696/1404] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:17.869735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502461e-924c-4832-b757-013bd747dee1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:17.870168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502461e-924c-4832-b757-013bd747dee1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:17.870351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502461e-924c-4832-b757-013bd747dee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:17.948034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d502461e-924c-4832-b757-013bd747dee1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:17.949269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d502461e-924c-4832-b757-013bd747dee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:18.064982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 709/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:18.358187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66821b38-ec37-48d6-a7da-cc94b0428893 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:18.359802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-66821b38-ec37-48d6-a7da-cc94b0428893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:18.516773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:18.532230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66821b38-ec37-48d6-a7da-cc94b0428893 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:18.659647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:18.660178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:18.660463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:18.660734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:18.662351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:18.762560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:18.764177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb8fb-327a-4524-ac2f-072ce901b279 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:18.774933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:18.786044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 697/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:21.074818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da90743-f2aa-49db-a314-b660ad566f05 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.074818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da90743-f2aa-49db-a314-b660ad566f05 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.074818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da90743-f2aa-49db-a314-b660ad566f05 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.081350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcab29-fcd0-403d-a536-a2873b8cc5af None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.081859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcab29-fcd0-403d-a536-a2873b8cc5af None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.081952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcab29-fcd0-403d-a536-a2873b8cc5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.084307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 710/1407] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:53:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:21.156679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-cdbcab29-fcd0-403d-a536-a2873b8cc5af None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:21.157839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-cdbcab29-fcd0-403d-a536-a2873b8cc5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:21.272458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 698/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:21.583418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da90743-f2aa-49db-a314-b660ad566f05 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:21.584954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3da90743-f2aa-49db-a314-b660ad566f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:21.588075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:21.765012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da90743-f2aa-49db-a314-b660ad566f05 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:21.847101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:21.894937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:21.895699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.896077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.896452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.898234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:22.004143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:22.005920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe2009-af95-4fee-a3ab-8a9f75bdd2e0 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:22.025095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 711/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:31.270872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=67 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:32.022736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=61 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:35.809311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9004ef-a769-4977-b782-0feb05ed83bb None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:35.809971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9004ef-a769-4977-b782-0feb05ed83bb None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:35.809971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9004ef-a769-4977-b782-0feb05ed83bb None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:35.811239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 699/1410] 10.222.0.23 () {54 vars in 918 bytes} [Sat Sep 10 20:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:35.816390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd388e0-f7ee-4dfa-8bb8-4e6a643130e1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:35.816796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd388e0-f7ee-4dfa-8bb8-4e6a643130e1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:35.816931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd388e0-f7ee-4dfa-8bb8-4e6a643130e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:35.892709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1cd388e0-f7ee-4dfa-8bb8-4e6a643130e1 None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:35.893937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1cd388e0-f7ee-4dfa-8bb8-4e6a643130e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:36.003128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 712/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Sat Sep 10 20:53:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:36.017255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d1cb4-1b7c-4965-9daf-7faa8651d924 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:36.017572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d1cb4-1b7c-4965-9daf-7faa8651d924 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:36.017855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d1cb4-1b7c-4965-9daf-7faa8651d924 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:36.019242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 700/1412] 10.222.0.23 () {54 vars in 918 bytes} [Sat Sep 10 20:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:36.023675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9118ee-5aa7-4234-b12b-4fc288cc4981 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:36.023987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9118ee-5aa7-4234-b12b-4fc288cc4981 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:36.024262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9118ee-5aa7-4234-b12b-4fc288cc4981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:36.109130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:36.111976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9b9118ee-5aa7-4234-b12b-4fc288cc4981 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:36.113755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9b9118ee-5aa7-4234-b12b-4fc288cc4981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:36.231757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 713/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Sat Sep 10 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:38.867659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002624-dda7-4460-be43-2050a50b97d1 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:38.868196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002624-dda7-4460-be43-2050a50b97d1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:38.868196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002624-dda7-4460-be43-2050a50b97d1 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:38.869704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:38.875893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb447d8-ad3a-4a63-b5e0-3e348cf8a167 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:38.876283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb447d8-ad3a-4a63-b5e0-3e348cf8a167 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:38.876447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb447d8-ad3a-4a63-b5e0-3e348cf8a167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:38.942486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9eb447d8-ad3a-4a63-b5e0-3e348cf8a167 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:38.943691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9eb447d8-ad3a-4a63-b5e0-3e348cf8a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:38.997696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:39.039750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:39.328235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f002624-dda7-4460-be43-2050a50b97d1 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:39.329872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7f002624-dda7-4460-be43-2050a50b97d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:39.476432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f002624-dda7-4460-be43-2050a50b97d1 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:39.549106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:39.589017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:39.589579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:39.589858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:39.590125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:39.591724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:39.680655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:39.682293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce9c32-012f-425d-b388-eec545e99a09 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:39.700676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 702/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:40.869765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c829f-e632-4438-9d7d-774d51787329 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:40.870310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c829f-e632-4438-9d7d-774d51787329 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:40.870310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c829f-e632-4438-9d7d-774d51787329 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:40.871703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:40.878238 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f449e2d-38dd-49a6-a072-66bb89ecddf4 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:40.878688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f449e2d-38dd-49a6-a072-66bb89ecddf4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:40.878793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f449e2d-38dd-49a6-a072-66bb89ecddf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:40.957196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5f449e2d-38dd-49a6-a072-66bb89ecddf4 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:40.958443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5f449e2d-38dd-49a6-a072-66bb89ecddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:40.986341 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:41.060707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:41.211772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c829f-e632-4438-9d7d-774d51787329 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:41.213205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-017c829f-e632-4438-9d7d-774d51787329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:41.359593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c829f-e632-4438-9d7d-774d51787329 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:41.399604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:41.471822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:41.472348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:41.472627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:41.472900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:41.474512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:41.562110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:41.563697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55a7a7-04eb-434d-b3db-c324927c254d service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:41.581695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 716/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:42.727082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4afca5-666b-4893-a899-aea86acb0628 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.727580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4afca5-666b-4893-a899-aea86acb0628 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.727688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4afca5-666b-4893-a899-aea86acb0628 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.728998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:42.735737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afce34-4d02-4f44-9ec6-1ef7753792c4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.736187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afce34-4d02-4f44-9ec6-1ef7753792c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.736279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afce34-4d02-4f44-9ec6-1ef7753792c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.790549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:42.812858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e9afce34-4d02-4f44-9ec6-1ef7753792c4 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:42.814088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e9afce34-4d02-4f44-9ec6-1ef7753792c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:42.912172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:43.063991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4afca5-666b-4893-a899-aea86acb0628 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:43.065667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ba4afca5-666b-4893-a899-aea86acb0628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:43.213552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4afca5-666b-4893-a899-aea86acb0628 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:43.225886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:43.326259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:43.326869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:43.327149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:43.327419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:43.328997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:43.416501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:43.418109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7ced-b935-4bc9-894d-a71ea67fd90b service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:43.436995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 705/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:44.565217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd474-046f-4f2d-b38c-bdd310d58ce2 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:44.565817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd474-046f-4f2d-b38c-bdd310d58ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:44.565817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd474-046f-4f2d-b38c-bdd310d58ce2 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:44.567152 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:44.573594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26285-8c0b-4f16-9737-0aeb950b4f99 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:44.574057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26285-8c0b-4f16-9737-0aeb950b4f99 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:44.574154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26285-8c0b-4f16-9737-0aeb950b4f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:44.594205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:44.652492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7ed26285-8c0b-4f16-9737-0aeb950b4f99 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:44.654372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7ed26285-8c0b-4f16-9737-0aeb950b4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:44.762198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:44.912102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfd474-046f-4f2d-b38c-bdd310d58ce2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:44.913690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-15bfd474-046f-4f2d-b38c-bdd310d58ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:45.023639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:45.059037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfd474-046f-4f2d-b38c-bdd310d58ce2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:45.177790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:45.178314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:45.178593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:45.178863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:45.180450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:45.256203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:45.271210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:45.272824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aeb519-61a3-4a2e-9468-e878d9a7516e service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:45.291316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 719/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:46.420618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8768-6ddc-47e5-9f93-b1534045a902 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.421226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8768-6ddc-47e5-9f93-b1534045a902 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.421226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8768-6ddc-47e5-9f93-b1534045a902 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.422532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:46.428664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072d316-6962-4086-8261-2d04c0a2371b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.429139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072d316-6962-4086-8261-2d04c0a2371b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.429245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072d316-6962-4086-8261-2d04c0a2371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.506367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1072d316-6962-4086-8261-2d04c0a2371b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:46.507495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1072d316-6962-4086-8261-2d04c0a2371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:46.605717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:46.756757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8768-6ddc-47e5-9f93-b1534045a902 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:46.758235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-481b8768-6ddc-47e5-9f93-b1534045a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:46.842407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:46.905201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8768-6ddc-47e5-9f93-b1534045a902 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:47.021078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:47.021649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:47.021934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:47.022211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:47.023806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:47.073742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:47.115429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:47.117037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3300790-dea8-4653-84b9-6b8f9b29489a service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:47.136562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 708/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:53:48.194731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c121-0c5c-4761-8c88-858c59e154ea None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:48.195261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c121-0c5c-4761-8c88-858c59e154ea None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:48.195261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c121-0c5c-4761-8c88-858c59e154ea None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:48.196536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:48.203410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddc57-522f-454d-8949-93ae6ec1d91f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:48.203859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddc57-522f-454d-8949-93ae6ec1d91f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:48.203964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddc57-522f-454d-8949-93ae6ec1d91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:48.280329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-628ddc57-522f-454d-8949-93ae6ec1d91f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:48.281584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-628ddc57-522f-454d-8949-93ae6ec1d91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:48.378282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:48.528090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c121-0c5c-4761-8c88-858c59e154ea None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:48.529698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9c83c121-0c5c-4761-8c88-858c59e154ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:48.580050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:48.676971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c121-0c5c-4761-8c88-858c59e154ea None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:48.789802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:48.790343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:48.790621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:48.790893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:48.792478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:48.799832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:48.881655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:48.883259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58b759-6523-4231-b7d0-d42adc43f5bb service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:48.902364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 722/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:49.950296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce9d0b-8832-459c-9520-4a0084987435 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:49.951324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce9d0b-8832-459c-9520-4a0084987435 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:49.951656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce9d0b-8832-459c-9520-4a0084987435 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:49.952945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:49.959365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf954d-03a2-4da8-800c-0c5adc984a61 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:49.959814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf954d-03a2-4da8-800c-0c5adc984a61 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:49.959909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf954d-03a2-4da8-800c-0c5adc984a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:50.038269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-05bf954d-03a2-4da8-800c-0c5adc984a61 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:50.039466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-05bf954d-03a2-4da8-800c-0c5adc984a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:50.106838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:50.138107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:50.287732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce9d0b-8832-459c-9520-4a0084987435 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:50.289421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b7ce9d0b-8832-459c-9520-4a0084987435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:50.306872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:50.436043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce9d0b-8832-459c-9520-4a0084987435 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:50.522955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:50.549780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:50.550320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:50.550630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:50.550873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:50.552552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:50.642730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:50.644324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c18c1f-9fe2-47fe-84ed-ca169ada7268 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:50.662730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 711/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:51.725657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b2b4d-ea90-4ec0-8c07-ae96021ef3e0 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:51.726055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b2b4d-ea90-4ec0-8c07-ae96021ef3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:51.726214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b2b4d-ea90-4ec0-8c07-ae96021ef3e0 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:51.727813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:51.734462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31237e12-c1cc-4784-8270-d4435200240c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:51.734939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31237e12-c1cc-4784-8270-d4435200240c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:51.735093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31237e12-c1cc-4784-8270-d4435200240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:51.815805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-31237e12-c1cc-4784-8270-d4435200240c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:51.817010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-31237e12-c1cc-4784-8270-d4435200240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:51.859047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:51.914059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:52.066553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b2b4d-ea90-4ec0-8c07-ae96021ef3e0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:52.068081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ba6b2b4d-ea90-4ec0-8c07-ae96021ef3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:52.215019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b2b4d-ea90-4ec0-8c07-ae96021ef3e0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:52.269891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:52.330280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:52.330816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:52.331096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:52.331370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:52.332952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:52.424513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:52.426150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c44aa-3808-4e87-8244-e3638ffd52b4 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:52.444131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 725/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:53.506708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98b31-0153-40ca-bbf9-b33cb17840b3 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:53.507281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98b31-0153-40ca-bbf9-b33cb17840b3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:53.507281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98b31-0153-40ca-bbf9-b33cb17840b3 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:53.508597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:53.515187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721d3ab-a9cb-456b-b24b-07b6f45714b8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:53.515599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721d3ab-a9cb-456b-b24b-07b6f45714b8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:53.515763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721d3ab-a9cb-456b-b24b-07b6f45714b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:53.709322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6721d3ab-a9cb-456b-b24b-07b6f45714b8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:53.710568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6721d3ab-a9cb-456b-b24b-07b6f45714b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:53.712754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:53.812751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:53.967495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98b31-0153-40ca-bbf9-b33cb17840b3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:53.969050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1ed98b31-0153-40ca-bbf9-b33cb17840b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:54.115367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98b31-0153-40ca-bbf9-b33cb17840b3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:54.143515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:54.230544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:54.231158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:54.231441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:54.231717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:54.233818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:54.322595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:54.324177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b40a9-d5bb-48ab-8f80-7d13639ee27e service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:54.343408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 714/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:56.499791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44718e-e63b-4f22-b516-b5c86e1744b4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:56.500356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44718e-e63b-4f22-b516-b5c86e1744b4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:56.500356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44718e-e63b-4f22-b516-b5c86e1744b4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:56.501727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:56.507196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a3953-661e-4423-b3b0-d1dbae12a4f2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:56.507611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a3953-661e-4423-b3b0-d1dbae12a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:56.507732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a3953-661e-4423-b3b0-d1dbae12a4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:56.549192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:56.577200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d92a3953-661e-4423-b3b0-d1dbae12a4f2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:56.578479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d92a3953-661e-4423-b3b0-d1dbae12a4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:56.675859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:56.960512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44718e-e63b-4f22-b516-b5c86e1744b4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:56.962089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fc44718e-e63b-4f22-b516-b5c86e1744b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:57.064217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44718e-e63b-4f22-b516-b5c86e1744b4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:57.096035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:57.137434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:57.138060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:57.138334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:57.138603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:57.140868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:57.251136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:57.252491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad950573-de07-48ad-bb2c-b729fd30c580 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:57.271501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 728/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:59.559885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0dc0-e998-45c9-b2d5-0149543c1d2a None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:59.560447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0dc0-e998-45c9-b2d5-0149543c1d2a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:59.560447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0dc0-e998-45c9-b2d5-0149543c1d2a None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:59.562369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:59.567698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a047-576e-4ede-80bf-6df1512fbe8e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:59.568115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a047-576e-4ede-80bf-6df1512fbe8e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:59.568245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a047-576e-4ede-80bf-6df1512fbe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:59.636221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-61c8a047-576e-4ede-80bf-6df1512fbe8e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:59.637415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-61c8a047-576e-4ede-80bf-6df1512fbe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:59.650663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:59.743248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:00.034231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0dc0-e998-45c9-b2d5-0149543c1d2a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:00.035780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-580a0dc0-e998-45c9-b2d5-0149543c1d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:00.154066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0dc0-e998-45c9-b2d5-0149543c1d2a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:00.167486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:00.229150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:00.229784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:00.230064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:00.230363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:00.231946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:00.338974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:00.340401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956b43d-28a4-43f7-bb22-cfd286d53f45 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:00.359256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 717/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:02.814160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19cae3-361b-441b-b70f-d36430b1d484 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:02.814715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19cae3-361b-441b-b70f-d36430b1d484 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:02.814715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19cae3-361b-441b-b70f-d36430b1d484 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:02.816040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:02.821406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616b7c-694b-4ede-8b01-afc558373bd3 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:02.821897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616b7c-694b-4ede-8b01-afc558373bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:02.821997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616b7c-694b-4ede-8b01-afc558373bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:02.870881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:02.890192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a9616b7c-694b-4ede-8b01-afc558373bd3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:02.891400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a9616b7c-694b-4ede-8b01-afc558373bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:02.988301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:03.272852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19cae3-361b-441b-b70f-d36430b1d484 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:03.274419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ee19cae3-361b-441b-b70f-d36430b1d484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:03.377279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19cae3-361b-441b-b70f-d36430b1d484 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:03.417952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:03.453477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:03.454087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:03.454373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:03.454649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:03.456237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:03.565158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:03.566629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad53365-a6e2-4e94-9ec3-4c0d4e1b8cec service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:03.584360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 731/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:05.860340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d4a9a-6b6a-41e6-b7e0-910c2b0cca2a None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.860872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d4a9a-6b6a-41e6-b7e0-910c2b0cca2a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.860872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d4a9a-6b6a-41e6-b7e0-910c2b0cca2a None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.862284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:05.867449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314841d7-df52-4ab9-891f-f6de59785b8e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.867909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314841d7-df52-4ab9-891f-f6de59785b8e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.868016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314841d7-df52-4ab9-891f-f6de59785b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.937662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-314841d7-df52-4ab9-891f-f6de59785b8e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:05.938872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-314841d7-df52-4ab9-891f-f6de59785b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:05.959119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:06.034524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:06.323836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d4a9a-6b6a-41e6-b7e0-910c2b0cca2a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:06.325361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ed5d4a9a-6b6a-41e6-b7e0-910c2b0cca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:06.426567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d4a9a-6b6a-41e6-b7e0-910c2b0cca2a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:06.446210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:06.499849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:06.500456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:06.500732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:06.500999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:06.502618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:06.611359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:06.612727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fcf7e-971f-41b0-b285-9edf1f76bcfe service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:06.631165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 720/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:08.901081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef9e20-d867-47a3-9a06-c45b3f080999 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.901528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef9e20-d867-47a3-9a06-c45b3f080999 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.901645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef9e20-d867-47a3-9a06-c45b3f080999 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.903029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:08.908259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0a713-64d5-4212-982a-17b2e7ef1145 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.908706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0a713-64d5-4212-982a-17b2e7ef1145 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.908813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0a713-64d5-4212-982a-17b2e7ef1145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.971985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:08.975252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a9a0a713-64d5-4212-982a-17b2e7ef1145 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:08.976826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a9a0a713-64d5-4212-982a-17b2e7ef1145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:09.073928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:09.379057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef9e20-d867-47a3-9a06-c45b3f080999 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:09.380579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0aef9e20-d867-47a3-9a06-c45b3f080999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:09.483510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef9e20-d867-47a3-9a06-c45b3f080999 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:09.534004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:09.558084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:09.558693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:09.558972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:09.559245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:09.560832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:09.669155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:09.670538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6db7c-bbc4-46fa-867b-b3871101a1d1 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:09.688043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 734/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:11.985393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372faef-ff49-4e8c-9fe1-4e013e5f4e6d None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:11.985937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372faef-ff49-4e8c-9fe1-4e013e5f4e6d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:11.986055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372faef-ff49-4e8c-9fe1-4e013e5f4e6d None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:11.987351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:11.992783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5521-49ec-4ec8-bdb1-b64b9913cefa None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:11.993216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5521-49ec-4ec8-bdb1-b64b9913cefa None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:11.993313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5521-49ec-4ec8-bdb1-b64b9913cefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:12.058976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-66dd5521-49ec-4ec8-bdb1-b64b9913cefa None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:12.060174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-66dd5521-49ec-4ec8-bdb1-b64b9913cefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:12.090934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:12.158877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:54:12.444512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372faef-ff49-4e8c-9fe1-4e013e5f4e6d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:12.446093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e372faef-ff49-4e8c-9fe1-4e013e5f4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:12.551342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372faef-ff49-4e8c-9fe1-4e013e5f4e6d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:12.701372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:12.748550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:12.749683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:12.749956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:12.750221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:12.751789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:12.860715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:12.862203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea4209-8f96-4dce-b3a7-94143e7773cf service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:12.880507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 723/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:15.114721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6ba83-e10f-494a-9100-cf9c162bfa2f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:15.115261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6ba83-e10f-494a-9100-cf9c162bfa2f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:15.115261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6ba83-e10f-494a-9100-cf9c162bfa2f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:15.116531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:15.121764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5db2-7ed8-440d-9cec-ce250bf9c912 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:15.122171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5db2-7ed8-440d-9cec-ce250bf9c912 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:15.122317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5db2-7ed8-440d-9cec-ce250bf9c912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:15.200519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-906e5db2-7ed8-440d-9cec-ce250bf9c912 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:15.201761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-906e5db2-7ed8-440d-9cec-ce250bf9c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:15.214129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:15.319080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:15.618565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6ba83-e10f-494a-9100-cf9c162bfa2f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:15.620286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0be6ba83-e10f-494a-9100-cf9c162bfa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:15.724620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6ba83-e10f-494a-9100-cf9c162bfa2f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:15.781296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:15.799064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:15.799663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:15.799940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:15.800208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:15.801800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:15.910647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:15.912091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb1dcd-99f6-483d-96c9-b47bd75c7fd6 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:15.929998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 737/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:18.164084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e9d34-67c9-4ee1-b3a7-ad2554a2c29e None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:18.164621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e9d34-67c9-4ee1-b3a7-ad2554a2c29e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:18.164621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e9d34-67c9-4ee1-b3a7-ad2554a2c29e None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:18.165962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:18.171935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99193-db10-4ae4-8411-504f20fc4208 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:18.172386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99193-db10-4ae4-8411-504f20fc4208 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:18.172526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99193-db10-4ae4-8411-504f20fc4208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:18.240104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b1e99193-db10-4ae4-8411-504f20fc4208 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:18.241326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b1e99193-db10-4ae4-8411-504f20fc4208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:18.280527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:18.339791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:18.623810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e9d34-67c9-4ee1-b3a7-ad2554a2c29e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:18.626159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-be0e9d34-67c9-4ee1-b3a7-ad2554a2c29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:18.729237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e9d34-67c9-4ee1-b3a7-ad2554a2c29e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:18.768542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:18.804004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:18.804611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:18.804892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:18.805164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:18.806789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:18.916892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:18.918372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159850-4539-440d-b6a5-1e3dd4c6c497 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:18.936241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 726/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:21.098673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ac861-552d-4262-a36e-6848f82b6d2f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:21.099203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ac861-552d-4262-a36e-6848f82b6d2f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:21.099203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ac861-552d-4262-a36e-6848f82b6d2f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:21.100549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:21.105748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323c7f-81a6-4387-83f7-36df8e7c58f5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:21.106193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323c7f-81a6-4387-83f7-36df8e7c58f5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:21.106329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323c7f-81a6-4387-83f7-36df8e7c58f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:21.172621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-16323c7f-81a6-4387-83f7-36df8e7c58f5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:21.173759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-16323c7f-81a6-4387-83f7-36df8e7c58f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:21.191749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:21.271961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:21.553851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ac861-552d-4262-a36e-6848f82b6d2f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:21.555397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fd5ac861-552d-4262-a36e-6848f82b6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:21.658074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ac861-552d-4262-a36e-6848f82b6d2f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:21.723276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:21.730252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:21.730851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:21.731126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:21.731391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:21.732950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:21.842398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:21.844376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2c196-6aaa-4845-b435-9467ade51dcc service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:21.863067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 740/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:24.018643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44d320-73a6-448d-8d3c-02dd870176eb None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:24.019264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44d320-73a6-448d-8d3c-02dd870176eb None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:24.019264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44d320-73a6-448d-8d3c-02dd870176eb None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:24.020467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:24.025939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad0531-1445-4cfb-b798-cfc770db8f0f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:24.026353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad0531-1445-4cfb-b798-cfc770db8f0f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:24.026474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad0531-1445-4cfb-b798-cfc770db8f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:24.094033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-78ad0531-1445-4cfb-b798-cfc770db8f0f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:24.095319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-78ad0531-1445-4cfb-b798-cfc770db8f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:24.143070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:24.192227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:24.487402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44d320-73a6-448d-8d3c-02dd870176eb None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:24.488926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4b44d320-73a6-448d-8d3c-02dd870176eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:24.596477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44d320-73a6-448d-8d3c-02dd870176eb None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:24.646026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:24.672038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:24.672649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:24.672924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:24.673190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:24.674800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:24.786982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:24.788413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1722ca2-00ef-424a-a74d-cb9e950cd568 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:24.806805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 729/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:34.190704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=22 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:34.804596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=66 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:44.865896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea386fe-366e-49de-a145-67f937947bc1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:44.866427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea386fe-366e-49de-a145-67f937947bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:44.866427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea386fe-366e-49de-a145-67f937947bc1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:44.867801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 742/1471] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:44.873103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bb15f-6b70-4a5f-a01c-1efb8629304e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:44.873560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bb15f-6b70-4a5f-a01c-1efb8629304e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:44.873731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bb15f-6b70-4a5f-a01c-1efb8629304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:44.941336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ea9bb15f-6b70-4a5f-a01c-1efb8629304e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:44.942614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ea9bb15f-6b70-4a5f-a01c-1efb8629304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:45.039960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 730/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:54:45.342556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269f15b-2277-4d70-be4c-5a7cddae1a1e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:45.342841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269f15b-2277-4d70-be4c-5a7cddae1a1e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:45.343113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269f15b-2277-4d70-be4c-5a7cddae1a1e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:45.344443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 743/1473] 10.222.0.23 () {54 vars in 966 bytes} [Sat Sep 10 20:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:45.349637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57824-4db0-4033-9831-996f0bfa2522 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:45.349930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57824-4db0-4033-9831-996f0bfa2522 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:45.350208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57824-4db0-4033-9831-996f0bfa2522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:45.426835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c2d57824-4db0-4033-9831-996f0bfa2522 None None] MFA Rules not processed for user `ee9a524cfaa141409ff24c149b698be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:45.428045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c2d57824-4db0-4033-9831-996f0bfa2522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:45.448547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:45.505443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 731/1474] 10.222.0.23 () {58 vars in 1063 bytes} [Sat Sep 10 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:45.513117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269f15b-2277-4d70-be4c-5a7cddae1a1e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:45.514924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e269f15b-2277-4d70-be4c-5a7cddae1a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:45.620228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269f15b-2277-4d70-be4c-5a7cddae1a1e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:45.713861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:45.714466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:45.714741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:45.715010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:45.717239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:45.772079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:45.826326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:45.827751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858ed6b-a5e1-48c6-8d21-2485f13f26a4 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:45.846730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 744/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:48.144919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79150627-560b-4389-9011-12f17ceb973c None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:48.145560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79150627-560b-4389-9011-12f17ceb973c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:48.145560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79150627-560b-4389-9011-12f17ceb973c None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:48.146850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 732/1476] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:48.152133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bfeb7-0005-4fef-a667-2f0dff41a5f9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:48.152560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bfeb7-0005-4fef-a667-2f0dff41a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:48.152664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bfeb7-0005-4fef-a667-2f0dff41a5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:48.218427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-563bfeb7-0005-4fef-a667-2f0dff41a5f9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:48.219752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-563bfeb7-0005-4fef-a667-2f0dff41a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:48.314954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 745/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:48.590517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79150627-560b-4389-9011-12f17ceb973c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:48.592041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-79150627-560b-4389-9011-12f17ceb973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:48.717334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79150627-560b-4389-9011-12f17ceb973c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:48.767812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:48.812191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:48.812799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:48.813128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:48.813759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:48.814196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:48.815961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:54:48.815961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:54:48.815961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:54:48.817111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8d5f9-afe6-4c0c-8e5a-ae2599c4da62 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:48.835985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 733/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Sat Sep 10 20:54:48 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:50.991955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200676-7366-41d5-8550-9b714e2302b8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:50.992524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200676-7366-41d5-8550-9b714e2302b8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:50.992524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200676-7366-41d5-8550-9b714e2302b8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:50.993943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 746/1479] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:50.999292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609032d1-be22-4780-bd4c-20972b19f9ba None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:50.999749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609032d1-be22-4780-bd4c-20972b19f9ba None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:50.999854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609032d1-be22-4780-bd4c-20972b19f9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:51.068988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-609032d1-be22-4780-bd4c-20972b19f9ba None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:51.070263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-609032d1-be22-4780-bd4c-20972b19f9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:51.167789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 734/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:54:51.458888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c200676-7366-41d5-8550-9b714e2302b8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:51.460417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3c200676-7366-41d5-8550-9b714e2302b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:51.470963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:51.607347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c200676-7366-41d5-8550-9b714e2302b8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:51.688053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:51.721926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:51.722464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:51.722742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:51.723012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:51.724598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:51.817214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:51.818834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d81f2-3b53-4306-9ee5-2571f94b315b service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:51.836788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 747/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:54.161155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44eaef-ee30-4d0e-bf31-84184e9c9087 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:54.161782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44eaef-ee30-4d0e-bf31-84184e9c9087 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:54.161782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44eaef-ee30-4d0e-bf31-84184e9c9087 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:54.163080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 735/1482] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:54.168293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356d56-9698-4cf3-bf2c-3e23df83f87e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:54.168703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356d56-9698-4cf3-bf2c-3e23df83f87e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:54.168837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356d56-9698-4cf3-bf2c-3e23df83f87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:54.233159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0d356d56-9698-4cf3-bf2c-3e23df83f87e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:54.234442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0d356d56-9698-4cf3-bf2c-3e23df83f87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:54.280261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:54.330950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 748/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:54.624319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44eaef-ee30-4d0e-bf31-84184e9c9087 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:54.626525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3b44eaef-ee30-4d0e-bf31-84184e9c9087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:54.650587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:54.729773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44eaef-ee30-4d0e-bf31-84184e9c9087 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:54.807674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:54.808281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:54.808562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:54.808835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:54.810450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:54.883643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:54.920252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:54.921714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee2283-6c09-4a38-bfdf-35e9c50b4abd service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:54.941152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 736/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:57.200765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106712ae-687c-441f-99db-929042b7ffef None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:57.201312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106712ae-687c-441f-99db-929042b7ffef None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:57.201312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106712ae-687c-441f-99db-929042b7ffef None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:57.202613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 749/1485] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:54:57.208607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b467d-14f8-4eb6-9db9-8b650cf5d41d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:57.209069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b467d-14f8-4eb6-9db9-8b650cf5d41d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:57.209180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b467d-14f8-4eb6-9db9-8b650cf5d41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:57.276566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1e1b467d-14f8-4eb6-9db9-8b650cf5d41d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:57.277826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1e1b467d-14f8-4eb6-9db9-8b650cf5d41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:57.374078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 737/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:57.664283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106712ae-687c-441f-99db-929042b7ffef None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:57.665854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-106712ae-687c-441f-99db-929042b7ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:57.771820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106712ae-687c-441f-99db-929042b7ffef None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:57.842726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:57.844242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:57.845516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:57.845793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:57.846263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:57.848245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:57.956746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:57.958140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a0c59-d7f9-43a1-b27b-a1eedccbef39 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:57.976217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 750/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:00.200891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96874f1c-024a-4a72-815a-57ae466e2965 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.201455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96874f1c-024a-4a72-815a-57ae466e2965 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.201455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96874f1c-024a-4a72-815a-57ae466e2965 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.202906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 738/1488] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:55:00.208376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b880763-6439-48c4-82e1-5ef640b63c47 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.208790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b880763-6439-48c4-82e1-5ef640b63c47 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.208949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b880763-6439-48c4-82e1-5ef640b63c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.275830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3b880763-6439-48c4-82e1-5ef640b63c47 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:00.277018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3b880763-6439-48c4-82e1-5ef640b63c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.328916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:00.373189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 751/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:00.672513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96874f1c-024a-4a72-815a-57ae466e2965 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.675023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-96874f1c-024a-4a72-815a-57ae466e2965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.711769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:00.780606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96874f1c-024a-4a72-815a-57ae466e2965 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.854268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.854873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.855151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.855424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.856998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.941875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:00.972371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.973991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255c543-9bd7-42e3-9be1-4779b1226552 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.992587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 739/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:03.377123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd67ef7-ac5f-4ad0-b8b9-8b4a9705e6c1 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:03.377694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd67ef7-ac5f-4ad0-b8b9-8b4a9705e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:03.377694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd67ef7-ac5f-4ad0-b8b9-8b4a9705e6c1 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:03.379102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 752/1491] 10.222.0.23 () {54 vars in 927 bytes} [Sat Sep 10 20:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:55:03.384537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be2d72-cffc-4150-bc5f-155e1b4a3b95 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:03.384930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be2d72-cffc-4150-bc5f-155e1b4a3b95 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:03.385080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be2d72-cffc-4150-bc5f-155e1b4a3b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:03.450201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-03be2d72-cffc-4150-bc5f-155e1b4a3b95 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:03.451391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-03be2d72-cffc-4150-bc5f-155e1b4a3b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:03.549317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 740/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Sat Sep 10 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:03.837266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd67ef7-ac5f-4ad0-b8b9-8b4a9705e6c1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:03.838875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3cd67ef7-ac5f-4ad0-b8b9-8b4a9705e6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:03.946407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd67ef7-ac5f-4ad0-b8b9-8b4a9705e6c1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:04.021967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:04.022579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:04.022858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:04.023132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:04.024757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:04.033799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:04.134836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:04.136279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67797f9f-bf54-4ea6-b7ce-e836d9b1ccff service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:04.155290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 753/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:13.547280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=82 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:14.153078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=50 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:17.885762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:17.886305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:17.886305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:17.962021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:17.963239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.062757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 741/1494] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:18.070801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.071336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.071440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.150457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:18.152404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.254798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 754/1495] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:18.259563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.260770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.272859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d6fc0-1d90-45f7-856c-3f1d1f8d9842 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.334488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:18.368161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.368767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.369045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.369316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.369992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:18.370291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.371291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:18.371291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:56:18.371291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:56:18.372172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.383444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 742/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:18.391826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.401936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fea54b-5d70-4784-8396-95e013c7d362 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.470117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:18.508074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.508693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.508971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.509243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.509919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.511680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:18.511680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:56:18.511680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:56:18.512847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.579461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 755/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:18.580649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.583267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.619529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcabd9-d7fc-4879-9267-1d6344ae6b82 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.714248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.714781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.715061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.715336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.715960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.716964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:18.716964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:56:18.716964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:56:18.718272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.762042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 743/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:18.766240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.767747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.829933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccd54c-40f8-4b9b-be14-47beb644dad3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.914632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:18.925997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.926612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.926893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.927164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.927795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.928803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:18.928803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:56:18.928803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:56:18.929721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.942846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 756/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:18.946924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.948481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.960718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627ad48-c7b7-4ade-b92a-d0b989edc21b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.985633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:19.055331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.055942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.056217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.056483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.057113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.058148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:19.058148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:56:19.058148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:56:19.059033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b037dd-acc3-4812-bc35-edcd47f63a85 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.071693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 744/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:19.075927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.077479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:19.110913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69977e1c-2756-4cda-9795-04de000bfce6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:19.227131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.227793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.228073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.228348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] PATH_INFO: `/v3/projects/d2fd02dff29d4af78f9e507fe650760f/users/e393fe47a8df434c9813f75eb4f01804/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.251114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fd02dff29d4af78f9e507fe650760f, user_id=e393fe47a8df434c9813f75eb4f01804, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.252072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:19.252072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:56:19.252072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:56:19.254801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e03d-afa5-4908-ad63-2c95fad4f8e6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.278682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 757/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:19 2022] PUT /identity/v3/projects/d2fd02dff29d4af78f9e507fe650760f/users/e393fe47a8df434c9813f75eb4f01804/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:19.290886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7651c-14aa-4c9f-8fc5-de1417f61a23 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.291173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7651c-14aa-4c9f-8fc5-de1417f61a23 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.291449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7651c-14aa-4c9f-8fc5-de1417f61a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.347943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-daa7651c-14aa-4c9f-8fc5-de1417f61a23 None None] MFA Rules not processed for user `e393fe47a8df434c9813f75eb4f01804`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:19.349141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-daa7651c-14aa-4c9f-8fc5-de1417f61a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:19.427688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 745/1502] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:19.441611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e9f10-dd91-4aea-a29e-b0f0ee11e2ac None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.441901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e9f10-dd91-4aea-a29e-b0f0ee11e2ac None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.442176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e9f10-dd91-4aea-a29e-b0f0ee11e2ac None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.443516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 758/1503] 10.222.0.23 () {54 vars in 964 bytes} [Sat Sep 10 20:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:19.448156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b909e-e596-4bc3-bab0-fe0f9b066b14 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.448443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b909e-e596-4bc3-bab0-fe0f9b066b14 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.448715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b909e-e596-4bc3-bab0-fe0f9b066b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.479237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:19.525355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a21b909e-e596-4bc3-bab0-fe0f9b066b14 None None] MFA Rules not processed for user `f07ba8c1958e4e2e82cdd4e490011593`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:19.526600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a21b909e-e596-4bc3-bab0-fe0f9b066b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:19.602676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 746/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Sat Sep 10 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:19.610978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e9f10-dd91-4aea-a29e-b0f0ee11e2ac None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.612668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0d7e9f10-dd91-4aea-a29e-b0f0ee11e2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:19.661684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:19.714936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e9f10-dd91-4aea-a29e-b0f0ee11e2ac None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:19.788631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.789162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.789436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.789736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.791308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:19.890328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.891773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.895600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:19.910187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 759/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:56:19.955501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5bfa0-ed3b-4002-adb1-0f3b9ddf2e56 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.955946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5bfa0-ed3b-4002-adb1-0f3b9ddf2e56 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.956151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5bfa0-ed3b-4002-adb1-0f3b9ddf2e56 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.957513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 747/1506] 10.222.0.23 () {52 vars in 830 bytes} [Sat Sep 10 20:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:19.961067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.962346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:20.060647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cc4d6-cf94-45d4-854c-178b518f9581 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.154301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.154967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.155207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.155477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] PATH_INFO: `/v3/users/e393fe47a8df434c9813f75eb4f01804` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.168595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e393fe47a8df434c9813f75eb4f01804)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.169705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:20.169705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:56:20.169705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:56:20.170909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.180568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:20.236667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e393fe47a8df434c9813f75eb4f01804'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:20.246645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e393fe47a8df434c9813f75eb4f01804'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:20.247293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e393fe47a8df434c9813f75eb4f01804'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:20.247826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e393fe47a8df434c9813f75eb4f01804'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:20.264002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e393fe47a8df434c9813f75eb4f01804'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:20.292371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 760/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:19 2022] DELETE /identity/v3/users/e393fe47a8df434c9813f75eb4f01804 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:20.296732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.297022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.297301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.355985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:20.357156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:20.432091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:20.444577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 748/1508] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:20.454259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.455462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:20.553364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12e0b6-1bed-43d2-9201-db29c9a4595d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.606976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:20.629717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.630334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.630613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.630911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.632494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:20.739567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.741006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a15668-8c53-4f84-85dd-921eb34912fc service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.759550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 761/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:21.109808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:21.111149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:21.123259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cde03e-d4fc-491d-bbba-51dc7001dc69 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:21.222437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:21.222970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:21.223244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:21.223511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] PATH_INFO: `/v3/projects/d2fd02dff29d4af78f9e507fe650760f` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:21.233189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fd02dff29d4af78f9e507fe650760f)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:21.234250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:56:21.234250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:56:21.234250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:56:21.235734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:21.295780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fd02dff29d4af78f9e507fe650760f', 'request_id': 'req-20c45b2f-5237-40a3-a318-aaa24a509247'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:21.309984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fd02dff29d4af78f9e507fe650760f', 'request_id': 'req-20c45b2f-5237-40a3-a318-aaa24a509247'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:21.318968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fd02dff29d4af78f9e507fe650760f', 'request_id': 'req-20c45b2f-5237-40a3-a318-aaa24a509247'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:21.342203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] The token cache is being invalidate because project d2fd02dff29d4af78f9e507fe650760f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:21.342493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:21.342924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-20c45b2f-5237-40a3-a318-aaa24a509247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:21.344345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 749/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:21 2022] DELETE /identity/v3/projects/d2fd02dff29d4af78f9e507fe650760f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:30.757314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=62 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:31.340156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:51.698284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11cfe8-6dd2-4883-aa42-ba40b8636352 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:51.698964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11cfe8-6dd2-4883-aa42-ba40b8636352 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:51.698964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11cfe8-6dd2-4883-aa42-ba40b8636352 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:51.700201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:57:51.705022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048025a-6381-48e6-96f9-ac426b27d920 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:51.705485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048025a-6381-48e6-96f9-ac426b27d920 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:51.705691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048025a-6381-48e6-96f9-ac426b27d920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:51.789001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1048025a-6381-48e6-96f9-ac426b27d920 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:51.790249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1048025a-6381-48e6-96f9-ac426b27d920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:51.896030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:52.049075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11cfe8-6dd2-4883-aa42-ba40b8636352 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:52.051263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1e11cfe8-6dd2-4883-aa42-ba40b8636352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:52.171470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11cfe8-6dd2-4883-aa42-ba40b8636352 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:52.177405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ad7e-ec9f-404f-bf84-3aaff2eadcda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:52.178035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ad7e-ec9f-404f-bf84-3aaff2eadcda service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:52.178315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ad7e-ec9f-404f-bf84-3aaff2eadcda service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:52.178595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ad7e-ec9f-404f-bf84-3aaff2eadcda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:52.186677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5ad7e-ec9f-404f-bf84-3aaff2eadcda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:52.188249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5ad7e-ec9f-404f-bf84-3aaff2eadcda service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:52.198055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 763/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:53.197520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e58c5d-4211-4642-943c-4442e07476ba None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:53.198079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e58c5d-4211-4642-943c-4442e07476ba None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:53.198079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e58c5d-4211-4642-943c-4442e07476ba None None] PATH_INFO: `/v3` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:53.199331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 751/1514] 10.222.0.44 () {54 vars in 925 bytes} [Sat Sep 10 20:57:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:57:53.221935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d0a34-b208-4c5f-b6dc-350d75e32256 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:53.222503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d0a34-b208-4c5f-b6dc-350d75e32256 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:53.222503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d0a34-b208-4c5f-b6dc-350d75e32256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:53.284322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6e6d0a34-b208-4c5f-b6dc-350d75e32256 None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:53.285627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6e6d0a34-b208-4c5f-b6dc-350d75e32256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:53.300398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5fe6-1aa3-4974-8273-f1aaf021c886 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:53.300688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5fe6-1aa3-4974-8273-f1aaf021c886 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:53.300969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5fe6-1aa3-4974-8273-f1aaf021c886 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:53.302756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 752/1515] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:57:53.308949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:53.309237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:53.309545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:53.331325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:53.333109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:53.341018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 753/1516] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:53.355254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:53.359446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 764/1517] 10.222.0.44 () {58 vars in 1015 bytes} [Sat Sep 10 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:53.496105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:53.508798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a653264-ca6d-4e82-8f32-4de86f7b785e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:53.514255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774e091-a8cb-4b57-b9ef-b018815b62a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:53.514848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e091-a8cb-4b57-b9ef-b018815b62a6 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:53.515128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e091-a8cb-4b57-b9ef-b018815b62a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:53.515401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e091-a8cb-4b57-b9ef-b018815b62a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:53.524578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774e091-a8cb-4b57-b9ef-b018815b62a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:53.526199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774e091-a8cb-4b57-b9ef-b018815b62a6 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:53.535635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 754/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:01.987590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882402d-5186-4f4a-80f2-d9a51e9957fc None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:01.988020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882402d-5186-4f4a-80f2-d9a51e9957fc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:01.988132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882402d-5186-4f4a-80f2-d9a51e9957fc None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:01.989510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 765/1519] 10.222.0.23 () {54 vars in 961 bytes} [Sat Sep 10 20:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:01.995182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19172f-fa1b-47d1-8d41-0a8367f28e93 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:01.995625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19172f-fa1b-47d1-8d41-0a8367f28e93 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:01.995723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19172f-fa1b-47d1-8d41-0a8367f28e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:02.015301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5f19172f-fa1b-47d1-8d41-0a8367f28e93 None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:02.016498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5f19172f-fa1b-47d1-8d41-0a8367f28e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:02.025920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 755/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Sep 10 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:02.033533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882402d-5186-4f4a-80f2-d9a51e9957fc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:02.047490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882402d-5186-4f4a-80f2-d9a51e9957fc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:02.052417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc3a68-32ff-43b4-b938-b57bbed0bbbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:02.053015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc3a68-32ff-43b4-b938-b57bbed0bbbe service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:02.053294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc3a68-32ff-43b4-b938-b57bbed0bbbe service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:02.053599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc3a68-32ff-43b4-b938-b57bbed0bbbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:02.061692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc3a68-32ff-43b4-b938-b57bbed0bbbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:02.063269 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc3a68-32ff-43b4-b938-b57bbed0bbbe service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:02.072759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 766/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:05.000213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2c532-22ed-4842-973f-5e574c43a8a0 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:05.000771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2c532-22ed-4842-973f-5e574c43a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:05.000771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2c532-22ed-4842-973f-5e574c43a8a0 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:05.002159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:05.008402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8261e-cb59-4c82-9db4-64fb278af051 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:05.008877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8261e-cb59-4c82-9db4-64fb278af051 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:05.008982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8261e-cb59-4c82-9db4-64fb278af051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:05.029597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-17a8261e-cb59-4c82-9db4-64fb278af051 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:05.030800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-17a8261e-cb59-4c82-9db4-64fb278af051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:05.039258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:05.191451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2c532-22ed-4842-973f-5e574c43a8a0 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:05.206992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2c532-22ed-4842-973f-5e574c43a8a0 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:05.213031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a415e9-5380-457f-9ebc-c2ffe69a1729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:05.213655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a415e9-5380-457f-9ebc-c2ffe69a1729 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:05.213937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a415e9-5380-457f-9ebc-c2ffe69a1729 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:05.214213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a415e9-5380-457f-9ebc-c2ffe69a1729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:05.222179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a415e9-5380-457f-9ebc-c2ffe69a1729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:05.223753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a415e9-5380-457f-9ebc-c2ffe69a1729 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:05.233330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 757/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:06.310087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71009d1c-ffcb-404b-90e8-dad01cac254d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:06.310635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71009d1c-ffcb-404b-90e8-dad01cac254d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:06.310635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71009d1c-ffcb-404b-90e8-dad01cac254d None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:06.312022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:06.318227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e47938-9bd7-4da6-8922-39a6d591fd20 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:06.318672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e47938-9bd7-4da6-8922-39a6d591fd20 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:06.318786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e47938-9bd7-4da6-8922-39a6d591fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:06.339473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b9e47938-9bd7-4da6-8922-39a6d591fd20 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:06.340975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b9e47938-9bd7-4da6-8922-39a6d591fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:06.349114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:06.501322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71009d1c-ffcb-404b-90e8-dad01cac254d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:06.516613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71009d1c-ffcb-404b-90e8-dad01cac254d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:06.522384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4329ee8-41ac-421e-a869-65390de8f4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:06.522981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329ee8-41ac-421e-a869-65390de8f4e5 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:06.523263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329ee8-41ac-421e-a869-65390de8f4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:06.523539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329ee8-41ac-421e-a869-65390de8f4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:06.531438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329ee8-41ac-421e-a869-65390de8f4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:06.533013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329ee8-41ac-421e-a869-65390de8f4e5 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:06.542544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 769/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:10.622809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83f59a-16e9-423b-8696-665965e4a381 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:10.623380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83f59a-16e9-423b-8696-665965e4a381 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:10.623380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83f59a-16e9-423b-8696-665965e4a381 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:10.624760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:10.630143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96dd9b-1a4a-43a7-a7e7-b974fe13c6f3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:10.630548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96dd9b-1a4a-43a7-a7e7-b974fe13c6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:10.630700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96dd9b-1a4a-43a7-a7e7-b974fe13c6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:10.651434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7f96dd9b-1a4a-43a7-a7e7-b974fe13c6f3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:10.652626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7f96dd9b-1a4a-43a7-a7e7-b974fe13c6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:10.660394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:10.950795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83f59a-16e9-423b-8696-665965e4a381 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:10.952481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7e83f59a-16e9-423b-8696-665965e4a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:10.979808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:11.034919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83f59a-16e9-423b-8696-665965e4a381 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:11.039327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61450509-39ee-43e7-b67d-57cff3ed91cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:11.039928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61450509-39ee-43e7-b67d-57cff3ed91cb service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:11.040206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61450509-39ee-43e7-b67d-57cff3ed91cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:11.040476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61450509-39ee-43e7-b67d-57cff3ed91cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:11.049377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61450509-39ee-43e7-b67d-57cff3ed91cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:11.050862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61450509-39ee-43e7-b67d-57cff3ed91cb service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:11.061070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 760/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:13.228445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40d64-39b1-47cc-a1c1-9366450cddfc None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:13.229020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40d64-39b1-47cc-a1c1-9366450cddfc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:13.229020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40d64-39b1-47cc-a1c1-9366450cddfc None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:13.230401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:13.235777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66256-d57f-4b18-9208-52f7af767c20 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:13.236240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66256-d57f-4b18-9208-52f7af767c20 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:13.236344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66256-d57f-4b18-9208-52f7af767c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:13.256015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-64b66256-d57f-4b18-9208-52f7af767c20 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:13.258055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-64b66256-d57f-4b18-9208-52f7af767c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:13.265828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:13.549010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40d64-39b1-47cc-a1c1-9366450cddfc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:13.560327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40d64-39b1-47cc-a1c1-9366450cddfc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:13.563845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d000-0f56-42bb-8436-b5f732399e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:13.564440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d000-0f56-42bb-8436-b5f732399e3e service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:13.564722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d000-0f56-42bb-8436-b5f732399e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:13.564999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d000-0f56-42bb-8436-b5f732399e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:13.573828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925d000-0f56-42bb-8436-b5f732399e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:13.575287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925d000-0f56-42bb-8436-b5f732399e3e service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:13.584609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 772/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:17.242268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613430d9-413e-486e-bc43-678ec6b2b18f None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:17.242802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613430d9-413e-486e-bc43-678ec6b2b18f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:17.242802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613430d9-413e-486e-bc43-678ec6b2b18f None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:17.244075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:17.249241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262d19f-8047-4565-8153-8652ff2b2813 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:17.249692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262d19f-8047-4565-8153-8652ff2b2813 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:17.249840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262d19f-8047-4565-8153-8652ff2b2813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:17.269721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e262d19f-8047-4565-8153-8652ff2b2813 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:17.270955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e262d19f-8047-4565-8153-8652ff2b2813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:17.279744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:17.582151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613430d9-413e-486e-bc43-678ec6b2b18f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:17.583803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-613430d9-413e-486e-bc43-678ec6b2b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:17.669974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613430d9-413e-486e-bc43-678ec6b2b18f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:17.673581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85083235-6f4d-4d6b-96a2-275dcd0f8a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:17.674187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85083235-6f4d-4d6b-96a2-275dcd0f8a72 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:17.674466 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85083235-6f4d-4d6b-96a2-275dcd0f8a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:17.674737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85083235-6f4d-4d6b-96a2-275dcd0f8a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:17.684334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85083235-6f4d-4d6b-96a2-275dcd0f8a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:17.685792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85083235-6f4d-4d6b-96a2-275dcd0f8a72 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:17.695995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 763/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:19.878178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4676c9-ec65-4000-86bb-a7bdd146fb95 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:19.878771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4676c9-ec65-4000-86bb-a7bdd146fb95 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:19.878771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4676c9-ec65-4000-86bb-a7bdd146fb95 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:19.880145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:58:19.885666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7807aec-6f94-40e9-8857-d021d10bb334 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:19.886112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7807aec-6f94-40e9-8857-d021d10bb334 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:19.886219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7807aec-6f94-40e9-8857-d021d10bb334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:19.906671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a7807aec-6f94-40e9-8857-d021d10bb334 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:19.907887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a7807aec-6f94-40e9-8857-d021d10bb334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:19.916307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:20.217236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4676c9-ec65-4000-86bb-a7bdd146fb95 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:20.218966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8f4676c9-ec65-4000-86bb-a7bdd146fb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:20.232125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4676c9-ec65-4000-86bb-a7bdd146fb95 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:20.235719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b2f6-4621-49be-820f-2d249f1ca60c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:20.236310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b2f6-4621-49be-820f-2d249f1ca60c service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:20.236585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b2f6-4621-49be-820f-2d249f1ca60c service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:20.236856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b2f6-4621-49be-820f-2d249f1ca60c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:20.246013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13b2f6-4621-49be-820f-2d249f1ca60c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:20.247435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13b2f6-4621-49be-820f-2d249f1ca60c service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:20.257231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 775/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:29.901687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=66 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:30.252574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=34 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:44.637008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038840-05e0-49fa-9eb5-328e37cbb6d5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.637649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038840-05e0-49fa-9eb5-328e37cbb6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.637649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038840-05e0-49fa-9eb5-328e37cbb6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.667489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a8038840-05e0-49fa-9eb5-328e37cbb6d5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.668742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a8038840-05e0-49fa-9eb5-328e37cbb6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.680541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 765/1540] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:44.689691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.690306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.690306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.699926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763b5b2-501e-42e7-ba00-64a09308d005 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.700266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763b5b2-501e-42e7-ba00-64a09308d005 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.700486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763b5b2-501e-42e7-ba00-64a09308d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.722448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.723689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.734585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f763b5b2-501e-42e7-ba00-64a09308d005 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.735813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f763b5b2-501e-42e7-ba00-64a09308d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.744701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 776/1541] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:44.749605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.750826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 766/1542] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:44.762705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.763034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.763258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.768095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a2875-8e24-44c3-baaa-fa650a8b5228 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.779154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.779786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.780067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.780453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.781148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:44.781544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.782577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:59:44.782577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:59:44.782577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:59:44.783471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424ce7f-d69b-497d-b9b3-caccc1a80f9c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.802562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.803809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.805497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 777/1543] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:59:44.810983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00095512-eff5-4ff6-891c-956782284f7d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.811275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00095512-eff5-4ff6-891c-956782284f7d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.811553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00095512-eff5-4ff6-891c-956782284f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.835243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 767/1544] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:44.836175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.844999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-00095512-eff5-4ff6-891c-956782284f7d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.846238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-00095512-eff5-4ff6-891c-956782284f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.852185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58f1b-3d9a-4545-98dc-4db73817596a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.872758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 778/1545] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:44.874688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00095512-eff5-4ff6-891c-956782284f7d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.888758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.889331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.889751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.890044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.890698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.891712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:59:44.891712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:59:44.891712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:59:44.892938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249967fa-92c6-4258-8f17-d97e27e783dd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.895675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00095512-eff5-4ff6-891c-956782284f7d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.918643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.919272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.919703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.919977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.920623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:44.920926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.922563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaffa5e-4489-4f6f-8daa-b43d5c531b1e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.947200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 779/1546] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:44.948664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345e21-7390-46cf-b588-563d7af6a2ac None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.948664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345e21-7390-46cf-b588-563d7af6a2ac None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.948664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345e21-7390-46cf-b588-563d7af6a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.979483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 768/1547] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:44.981900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.984430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.985040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.995138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8a345e21-7390-46cf-b588-563d7af6a2ac None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.997771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8a345e21-7390-46cf-b588-563d7af6a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.010996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.012258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.063654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 780/1548] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.064919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.065207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.065515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.081034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 769/1549] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.081915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.085581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.086786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.095237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 781/1550] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.095992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671c740-f633-4489-a878-4a5ef9c72cfc None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.096120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.100712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8847da-9d63-47c7-a889-806ee55e8d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.101329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8847da-9d63-47c7-a889-806ee55e8d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.101646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8847da-9d63-47c7-a889-806ee55e8d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.101925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8847da-9d63-47c7-a889-806ee55e8d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.102725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8847da-9d63-47c7-a889-806ee55e8d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.105288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8847da-9d63-47c7-a889-806ee55e8d1a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.106929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b5c7d-808c-449a-b6d9-7c81f43b3174 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.112045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.112642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.112919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.113191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.113978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.115420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:59:45.115420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:59:45.115420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:59:45.117019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11ba52-c0ee-4842-975c-2635f0ac68e0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.136198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 770/1551] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.137993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc7ab0-0458-4fdc-930a-cb71aa9839b1 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.137993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc7ab0-0458-4fdc-930a-cb71aa9839b1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.138213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc7ab0-0458-4fdc-930a-cb71aa9839b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.145770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 782/1552] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/users => generated 284 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.145770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.145770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.145770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.158426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-58dc7ab0-0458-4fdc-930a-cb71aa9839b1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.159654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-58dc7ab0-0458-4fdc-930a-cb71aa9839b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.168287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.169497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.220537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 771/1553] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.221825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e2c1c-00bb-4ca8-bf85-ef3446fe38d8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.222113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e2c1c-00bb-4ca8-bf85-ef3446fe38d8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.222392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e2c1c-00bb-4ca8-bf85-ef3446fe38d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.235810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 783/1554] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.244173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.244366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-428e2c1c-00bb-4ca8-bf85-ef3446fe38d8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.244669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-428e2c1c-00bb-4ca8-bf85-ef3446fe38d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.246928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46bfb8-5e6f-4fd6-8cb0-8798b555c98f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.251470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.252067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.252344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.252617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.252796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 772/1555] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.253389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:45.253723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.254067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.254348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.254625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.255468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3697d5e-0722-49db-b608-23737b90ff61 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.267668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 784/1556] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:45.268880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128316-fd5c-48da-8860-69356dff9404 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.269167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128316-fd5c-48da-8860-69356dff9404 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.269440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128316-fd5c-48da-8860-69356dff9404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.281215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.282509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.289117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-45128316-fd5c-48da-8860-69356dff9404 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.290813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-45128316-fd5c-48da-8860-69356dff9404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.291802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 773/1557] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.292502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.298813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 785/1558] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.299644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45128316-fd5c-48da-8860-69356dff9404 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.307934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f856bb-194d-4d4b-94f9-ea1290b3b0f8 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.309934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45128316-fd5c-48da-8860-69356dff9404 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.312611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff18208-cb4e-48db-b3ba-3025a84cb5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.313210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff18208-cb4e-48db-b3ba-3025a84cb5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.313511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff18208-cb4e-48db-b3ba-3025a84cb5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.313786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff18208-cb4e-48db-b3ba-3025a84cb5b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.314440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff18208-cb4e-48db-b3ba-3025a84cb5b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.314553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9246e-87c0-4687-927b-618b1eebdefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.315030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9246e-87c0-4687-927b-618b1eebdefb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.315305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9246e-87c0-4687-927b-618b1eebdefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.315574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9246e-87c0-4687-927b-618b1eebdefb admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.316193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9246e-87c0-4687-927b-618b1eebdefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.316351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff18208-cb4e-48db-b3ba-3025a84cb5b2 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.317805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9246e-87c0-4687-927b-618b1eebdefb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.328390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 786/1559] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:45.329902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b4170-dde4-48ba-88e7-fa931d3bbbab None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.330273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b4170-dde4-48ba-88e7-fa931d3bbbab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.330591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b4170-dde4-48ba-88e7-fa931d3bbbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.333867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:45.341572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 774/1560] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.343316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.343604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.343884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.349196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:45.355093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-487b4170-dde4-48ba-88e7-fa931d3bbbab None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.356633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-487b4170-dde4-48ba-88e7-fa931d3bbbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.364613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 787/1561] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.364987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.365869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.366153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.366266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.366956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.375565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 775/1562] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.376332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.386363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cc3d0-51d5-48ba-81b7-684b99986bff None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.386547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:45.387750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.390882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.391476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.391817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.392093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.392717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.394636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.395921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 788/1563] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.396658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.407498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb177b-12f8-4d01-9b0b-7e35c280935a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.412131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.412827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.413155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.413547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.414252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:45.414593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.417371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.423216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 776/1564] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.426830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.429177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 789/1565] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:45.436331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.440326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c0217-ab32-412a-b053-2c176e148735 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.490917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e741d7-0be8-4348-a1ac-03fb0cbacd99 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.497105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.497862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.498223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.498534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.499270 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:45.499665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.501273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.501550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.502164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.502443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.502753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.503374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.504953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.514988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 777/1566] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:45.515907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.516775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 790/1567] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:45.517634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.531715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5664e-ba7d-4567-a51f-24ce74de1d65 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.531942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8942e8-bbab-4462-974e-47cb15470bf0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.536252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.536466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.536844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.537049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.537153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.537334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.537436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.537643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.538030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.538274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:45.538603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.539623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.540690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.550781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 778/1568] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:45.551637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.560859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 791/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:45.560859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.562789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b28d9a-9010-475b-8120-77bfacd32b59 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.564973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329abce-d8e4-448f-910e-ff2a36aa3cb8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.567359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.567947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.568225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.568496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.569129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:45.569427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.569559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.570110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.570392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.570664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.571064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.571291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:45.571934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.573631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.582749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 779/1570] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:45.583581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.592738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 792/1571] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:45.592738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.595673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb926a-f2a8-4ab0-bb10-91e6efe2806a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.600229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.600841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.601119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.601391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.601975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6afe8-a13d-4811-978a-9d71d2629509 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.602094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.604475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.607291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.607877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.608155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.608427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.609041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.610949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.629327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 780/1572] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.629327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.637704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00bb0b-db32-4583-80c5-b25f77422f1c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.643554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.644508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.644626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 793/1573] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.644746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.644848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] PATH_INFO: `/v3/projects/479ec0254e1a4edeb48bda9a262981b3/users/33a6c442d9f34521b7594a476741bfdd/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.645192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.661957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=479ec0254e1a4edeb48bda9a262981b3, user_id=33a6c442d9f34521b7594a476741bfdd, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.662910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:59:45.662910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:59:45.662910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:59:45.664882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.678192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 781/1574] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:45 2022] PUT /identity/v3/projects/479ec0254e1a4edeb48bda9a262981b3/users/33a6c442d9f34521b7594a476741bfdd/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:45.689552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.719241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13897f16-83be-40bb-a79b-c2af8c3c7389 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.738974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:45.743996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9e92a-11a5-4180-8255-c311db93e95e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.749800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.750439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.750877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.751157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.751827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.752653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:45.753553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.764944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 782/1575] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:45.765613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.776387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14052e1-7bf3-4513-8f7f-447fcb4c7a8b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.781034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.781643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.781929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.782203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.782817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.783084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.783708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.783993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.784284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.784708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.784922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.786871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.813403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 783/1576] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.817514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.817644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 794/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 389 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.817644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.879638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf8f44-6d9e-40d3-a5ce-75adc9211e6e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.879796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa61007-554e-43a4-b358-b1bb2a293a9f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.884761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.885242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.885374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.885374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.885581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.885689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.885864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.886150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.886358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.886864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.888297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.888769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.917842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 795/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.918262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 784/1579] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:45.921904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.922075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.984099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90483d7-a14d-46f1-8c80-8964942543e3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.986740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72308d-f8af-4475-9705-f4a156fa3fea admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.988641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7d88-34d3-43d5-90e4-86ff48e380ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.989249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7d88-34d3-43d5-90e4-86ff48e380ff admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.989561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7d88-34d3-43d5-90e4-86ff48e380ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.989839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7d88-34d3-43d5-90e4-86ff48e380ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.990476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7d88-34d3-43d5-90e4-86ff48e380ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.991439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.992050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.992160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7d88-34d3-43d5-90e4-86ff48e380ff admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.992335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.992616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.993255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.995449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.002010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 796/1580] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.003187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.003475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.003748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.022953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 785/1581] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:45 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:46.022953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.031022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60cf31-e6b0-48e0-9d16-c812264a7df7 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.034208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] MFA Rules not processed for user `33a6c442d9f34521b7594a476741bfdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:46.035419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:46.035854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.036509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.036790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.037063 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] PATH_INFO: `/v3/projects/97165e035a134b5c9178f531801a533c/users/12f441d491f846018630b0871734aab0/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.053487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97165e035a134b5c9178f531801a533c, user_id=12f441d491f846018630b0871734aab0, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.056736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.067607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:46.070615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 786/1582] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/97165e035a134b5c9178f531801a533c/users/12f441d491f846018630b0871734aab0/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:46.075462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.100251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:46.104483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 797/1583] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:46.108630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.120146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759cd0d-c279-4494-a169-799a457cdba2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.149502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2b8e-3f99-44d1-8d64-53db9ad90088 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.154470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.155152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.155440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.155719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.156351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.158331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.178727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.179252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.179533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.179813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.180125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 787/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:46.180440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.181495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.183785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.193087 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6d871-52ff-4af6-bf46-77f25010e986 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.198242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.199013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.199301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.199576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.200324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.202442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.206489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 798/1585] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:46.207422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.221030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36b97a-7629-4e76-91b3-49619a98aa8a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.228414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 788/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:46.228414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.229316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.229966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.230249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.230523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.231149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.233085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.238734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ad62d-015e-48bd-ae45-a1670d83302e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.244880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd41d1-76fa-4b59-8405-402e2469bcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.245513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd41d1-76fa-4b59-8405-402e2469bcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.245805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd41d1-76fa-4b59-8405-402e2469bcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.246113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd41d1-76fa-4b59-8405-402e2469bcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.246733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd41d1-76fa-4b59-8405-402e2469bcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.248507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd41d1-76fa-4b59-8405-402e2469bcc9 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.256925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 799/1587] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:46.257898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.263342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 789/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.263342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.263342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.263342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.269026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1ae71-2431-43b4-ba0b-aed43721d3a7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.274322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.275606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.275973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.276322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.276944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.278727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.282559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:46.283784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:46.289966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 800/1589] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.292034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.292300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 790/1590] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:46.295676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.303595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bae937-1227-4a80-b73e-6719d42e94d2 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.309072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.309711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.310005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.310265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.311039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.312771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.323864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 791/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.324750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.334197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a927-d6d6-4fff-b2d0-f32aae785dee admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.339513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.340264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.340639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.341008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.341772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.343516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.352138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526ea7-7368-46e4-ae16-75da416ff257 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.353778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 792/1592] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.354572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.357177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.357828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.358111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.358389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.365000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c87333-435d-4ee9-8350-f7e1b586725c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.370198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.370790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.371076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.371351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.371980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.373596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.384302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 793/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.385195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.395552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79287a57-6191-4650-b75e-e44ce262eaf6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.401321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.402000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.402304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.402565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.403207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.404860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.414997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 794/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.416480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:46.416654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.427560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304a4d5-4960-4f0d-9491-3b8aaadb2705 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.432572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.433171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.433583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.433831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] PATH_INFO: `/v3/projects/5704e83d2f7243c09ab2b1bb14fb03a5/users/a0e373c5a6de4f82b29fcfae93e982cd/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.434836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.437167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.448657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5704e83d2f7243c09ab2b1bb14fb03a5, user_id=a0e373c5a6de4f82b29fcfae93e982cd, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.450446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 801/1595] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:46.451469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.451656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.467034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ae16-4250-4c27-befe-9ae598110d86 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.474577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 795/1596] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/5704e83d2f7243c09ab2b1bb14fb03a5/users/a0e373c5a6de4f82b29fcfae93e982cd/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:46.474577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.536706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4d4b-b62c-44bd-8f52-1df7f28796e0 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.541352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.542219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.542529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.542807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.543395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.544148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.544437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.544712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.545710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.547534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.552400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.553861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.559831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 802/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.561119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.567223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 796/1598] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:46.568054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.571841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa7a8f-9e5c-4cf1-89e3-23b6230c9dba admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.578305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.578990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.579233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ca82-bd46-4cf3-a262-2ebb28c7c5d5 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.579374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.579867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.580580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.582703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.585091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.585743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.586026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.586298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.587530 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.589256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.595511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:46.598591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 803/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.599882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.609512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 797/1600] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.610626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.620131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dea199-0690-426e-8a6e-636e92d613ae admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.624513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be243db-aa24-43c6-84fa-add82da6cf43 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.626460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61b4f6-0e7b-40f2-af4e-5ab1c7bc49db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.627082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61b4f6-0e7b-40f2-af4e-5ab1c7bc49db admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.627377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61b4f6-0e7b-40f2-af4e-5ab1c7bc49db admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.627639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61b4f6-0e7b-40f2-af4e-5ab1c7bc49db admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.628710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61b4f6-0e7b-40f2-af4e-5ab1c7bc49db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.630871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61b4f6-0e7b-40f2-af4e-5ab1c7bc49db admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.631019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.631632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.631915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.632494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.633320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.635699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.644619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 804/1601] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.646295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.646756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.647227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.653811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 798/1602] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:46.654756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.669443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dad69a-9afd-4be6-88aa-3ec48420bc50 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.675577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.676473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.676914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.677272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] PATH_INFO: `/v3/projects/1373d115e5ba401d96f2b60357e4d2b5/users/e1b96c72cc9a4af6a6ee540cf11e4f1e/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.692099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] MFA Rules not processed for user `a0e373c5a6de4f82b29fcfae93e982cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:46.693616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:46.703033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1373d115e5ba401d96f2b60357e4d2b5, user_id=e1b96c72cc9a4af6a6ee540cf11e4f1e, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.706574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.732364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 799/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/1373d115e5ba401d96f2b60357e4d2b5/users/e1b96c72cc9a4af6a6ee540cf11e4f1e/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:46.733082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.799870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 805/1604] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:46.805832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.819059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e37a3-9e60-4490-910b-7623950f0b7b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.827104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.827801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.828080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.828467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] PATH_INFO: `/v3/projects/c82c6a58da4641c18a387388d3b3ad73/users/a22c6bd0fac8443c844d7734aba97ef4/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.829192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629f8d-175d-4036-abb5-cafd19452605 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.835380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.836332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.836897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.837998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] PATH_INFO: `/v3/projects/c38e86c82baa44028fd2dfafbb49f42c/users/6e24c7d558f74ce181aad463bf18ee52/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.847359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82c6a58da4641c18a387388d3b3ad73, user_id=a22c6bd0fac8443c844d7734aba97ef4, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.850775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.859693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38e86c82baa44028fd2dfafbb49f42c, user_id=6e24c7d558f74ce181aad463bf18ee52, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:46.863010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:46.878654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 806/1605] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/c82c6a58da4641c18a387388d3b3ad73/users/a22c6bd0fac8443c844d7734aba97ef4/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:46.878654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.881039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 800/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/c38e86c82baa44028fd2dfafbb49f42c/users/6e24c7d558f74ce181aad463bf18ee52/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:46.890912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:46.931299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:46.983052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670e838-e565-4302-9f0e-1b1ff27333f0 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.986474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e899f89-4f3b-4c13-82b0-6bd724c8823b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:46.991620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3676f9-13bb-4b3c-b25d-2e82a682929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.992215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddcfca-2833-47e6-9ec5-2ffe3547807b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:46.992414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3676f9-13bb-4b3c-b25d-2e82a682929d admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.992898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddcfca-2833-47e6-9ec5-2ffe3547807b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:46.993016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3676f9-13bb-4b3c-b25d-2e82a682929d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.993217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddcfca-2833-47e6-9ec5-2ffe3547807b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:46.993322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3676f9-13bb-4b3c-b25d-2e82a682929d admin admin] PATH_INFO: `/v3/projects/12481b41ce3541eeb05a8a1722d9acbf/users/47f9883392364d3fb5d0b921d6f451d7/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:46.993627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddcfca-2833-47e6-9ec5-2ffe3547807b admin admin] PATH_INFO: `/v3/projects/4fc4783cb8fe4ef6a78e4c541a408c9a/users/16610ae2c3214396b78fb1caff01ab6e/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.010738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddcfca-2833-47e6-9ec5-2ffe3547807b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc4783cb8fe4ef6a78e4c541a408c9a, user_id=16610ae2c3214396b78fb1caff01ab6e, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.011918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3676f9-13bb-4b3c-b25d-2e82a682929d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12481b41ce3541eeb05a8a1722d9acbf, user_id=47f9883392364d3fb5d0b921d6f451d7, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.013705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddcfca-2833-47e6-9ec5-2ffe3547807b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.016559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3676f9-13bb-4b3c-b25d-2e82a682929d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.032674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 807/1607] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/4fc4783cb8fe4ef6a78e4c541a408c9a/users/16610ae2c3214396b78fb1caff01ab6e/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:47.036670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c928c6-3555-4b84-abbc-a0d4e8bae1e2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.036670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c928c6-3555-4b84-abbc-a0d4e8bae1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.036670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c928c6-3555-4b84-abbc-a0d4e8bae1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.046141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 801/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:46 2022] PUT /identity/v3/projects/12481b41ce3541eeb05a8a1722d9acbf/users/47f9883392364d3fb5d0b921d6f451d7/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:47.047992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3392b2-7ac6-4236-af32-9b1b75bc43c8 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.048491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3392b2-7ac6-4236-af32-9b1b75bc43c8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.048873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3392b2-7ac6-4236-af32-9b1b75bc43c8 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.052309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 802/1609] 10.222.0.23 () {54 vars in 962 bytes} [Sat Sep 10 20:59:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:59:47.054628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a9eaf-1fe9-43a4-8d29-1f5709cf4b45 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.054970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a9eaf-1fe9-43a4-8d29-1f5709cf4b45 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.055261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a9eaf-1fe9-43a4-8d29-1f5709cf4b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.090724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-57c928c6-3555-4b84-abbc-a0d4e8bae1e2 None None] MFA Rules not processed for user `e1b96c72cc9a4af6a6ee540cf11e4f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.093088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-57c928c6-3555-4b84-abbc-a0d4e8bae1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.112363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b61a9eaf-1fe9-43a4-8d29-1f5709cf4b45 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.112363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b61a9eaf-1fe9-43a4-8d29-1f5709cf4b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.151461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:47.202662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 803/1610] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.204082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca6c3d-b617-476b-90b6-02382ed3bcce None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.204378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca6c3d-b617-476b-90b6-02382ed3bcce None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.204656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca6c3d-b617-476b-90b6-02382ed3bcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.228125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 808/1611] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.229753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f91f-67f1-4f91-a072-0dd504892189 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.230439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f91f-67f1-4f91-a072-0dd504892189 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.231203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f91f-67f1-4f91-a072-0dd504892189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.231719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-46ca6c3d-b617-476b-90b6-02382ed3bcce None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.233534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-46ca6c3d-b617-476b-90b6-02382ed3bcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.267810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 804/1612] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.269246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a238ba6-7101-4058-b854-397344874af7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.269541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a238ba6-7101-4058-b854-397344874af7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.269822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a238ba6-7101-4058-b854-397344874af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.293491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7495f91f-67f1-4f91-a072-0dd504892189 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.294849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7495f91f-67f1-4f91-a072-0dd504892189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.297806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4a238ba6-7101-4058-b854-397344874af7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.299519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4a238ba6-7101-4058-b854-397344874af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.304313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 809/1613] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.306492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bc1a5-b2e2-45fa-9813-09d142e9023d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.306839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bc1a5-b2e2-45fa-9813-09d142e9023d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.307116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bc1a5-b2e2-45fa-9813-09d142e9023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.320669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 805/1614] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.322486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a238ba6-7101-4058-b854-397344874af7 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:47.355913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-290bc1a5-b2e2-45fa-9813-09d142e9023d None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.357218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-290bc1a5-b2e2-45fa-9813-09d142e9023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.408847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a238ba6-7101-4058-b854-397344874af7 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:47.416287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:47.417036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.417321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.417628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.438331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.440524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.450099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 810/1615] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.456960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 806/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:47.458345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:47.459717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018743f-2516-45ca-9acc-b4487ee85636 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.460012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018743f-2516-45ca-9acc-b4487ee85636 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.460288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018743f-2516-45ca-9acc-b4487ee85636 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.467743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 811/1617] 10.222.0.23 () {54 vars in 962 bytes} [Sat Sep 10 20:59:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:59:47.468549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6718caf-987b-4768-a98b-68e7cb6e92fe service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:47.468684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018743f-2516-45ca-9acc-b4487ee85636 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:47.473162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:47.473886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.474268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.474548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.480806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018743f-2516-45ca-9acc-b4487ee85636 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:47.484651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:47.485648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.485933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.486209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.488311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.490440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.498209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.500271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.503751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 807/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:47.504660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:47.505441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:47.515336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd4239-4cf9-4d25-9158-5855c417514d service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:47.519196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000baf3-bcc4-472b-abad-4a6d9818ce8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:47.520442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000baf3-bcc4-472b-abad-4a6d9818ce8d service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.520610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 812/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:47.520771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000baf3-bcc4-472b-abad-4a6d9818ce8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.521058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000baf3-bcc4-472b-abad-4a6d9818ce8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.521891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:47.532305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000baf3-bcc4-472b-abad-4a6d9818ce8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.534079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000baf3-bcc4-472b-abad-4a6d9818ce8d service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.541948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad93342-c17e-4db6-b322-7e1baad86d7a service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:47.545451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 808/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:47.546827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.547074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.547356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.550971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15352f46-cde2-41e2-babf-31135fcdb317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:47.552189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15352f46-cde2-41e2-babf-31135fcdb317 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.552757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15352f46-cde2-41e2-babf-31135fcdb317 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.553264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15352f46-cde2-41e2-babf-31135fcdb317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.570949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:47.572922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:47.583889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 809/1621] 10.222.0.23 () {58 vars in 1059 bytes} [Sat Sep 10 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:47.599961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:47.614691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19376b-86ba-47b1-9a89-d91176a1fce3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:47.622049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b190b0-b032-451d-9db8-5235bd5be4d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:47.622885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b190b0-b032-451d-9db8-5235bd5be4d3 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:47.623181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b190b0-b032-451d-9db8-5235bd5be4d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:47.623505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b190b0-b032-451d-9db8-5235bd5be4d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:47.635798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b190b0-b032-451d-9db8-5235bd5be4d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.637570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b190b0-b032-451d-9db8-5235bd5be4d3 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.650364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15352f46-cde2-41e2-babf-31135fcdb317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:47.651824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 810/1622] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:47.666993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15352f46-cde2-41e2-babf-31135fcdb317 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:47.713628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 813/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:48.106509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897abcb7-df6b-40bc-a094-ead92106a210 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:48.107197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897abcb7-df6b-40bc-a094-ead92106a210 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:48.107197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897abcb7-df6b-40bc-a094-ead92106a210 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:48.108737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 811/1624] 10.222.0.23 () {54 vars in 912 bytes} [Sat Sep 10 20:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:59:48.132292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46973004-de39-4f06-b39f-8384f38c6a91 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:48.132827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46973004-de39-4f06-b39f-8384f38c6a91 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:48.133048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46973004-de39-4f06-b39f-8384f38c6a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:48.166541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-46973004-de39-4f06-b39f-8384f38c6a91 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:48.168404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-46973004-de39-4f06-b39f-8384f38c6a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:48.196055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 814/1625] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 20:59:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:48.204027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897abcb7-df6b-40bc-a094-ead92106a210 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:48.252760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fef0f-604d-43cd-b7ea-1c81f7c31345 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:48.253158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fef0f-604d-43cd-b7ea-1c81f7c31345 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:48.253569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fef0f-604d-43cd-b7ea-1c81f7c31345 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:48.259489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 815/1626] 10.222.0.23 () {54 vars in 912 bytes} [Sat Sep 10 20:59:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:59:48.267653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:48.269272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:48.269272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:48.307339 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:48.309821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:48.339584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 816/1627] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 20:59:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:48.346259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897abcb7-df6b-40bc-a094-ead92106a210 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:48.347358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:48.350227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:48.350849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:48.351127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:48.351468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:48.361902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e696d1-a7a7-4765-a4b2-c8ed672ce1d3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:48.366071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:48.368032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:48.369855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1b980-bd04-4533-9db3-0953da449663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:48.371377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b980-bd04-4533-9db3-0953da449663 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:48.371377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b980-bd04-4533-9db3-0953da449663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:48.371645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b980-bd04-4533-9db3-0953da449663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:48.383079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 812/1628] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:59:48.403994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1b980-bd04-4533-9db3-0953da449663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:48.406625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1b980-bd04-4533-9db3-0953da449663 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:48.446099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 817/1629] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:49.116103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:49.139674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9852-e2df-44c6-b8f3-6d8beb61b5ea service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:49.161694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:49.164384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:49.164822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:49.165279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] PATH_INFO: `/v3/users/e1b96c72cc9a4af6a6ee540cf11e4f1e` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:49.173435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b96c72cc9a4af6a6ee540cf11e4f1e)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:49.174860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: WARNING py.warnings [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:59:49.174860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: warnings.warn(msg) Sep 10 20:59:49.174860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: Sep 10 20:59:49.176592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:49.210819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:49.300387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b96c72cc9a4af6a6ee540cf11e4f1e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:49.324786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b96c72cc9a4af6a6ee540cf11e4f1e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:49.326131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b96c72cc9a4af6a6ee540cf11e4f1e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:49.367919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b96c72cc9a4af6a6ee540cf11e4f1e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:49.419845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b96c72cc9a4af6a6ee540cf11e4f1e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:49.434819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 813/1630] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:49 2022] DELETE /identity/v3/users/e1b96c72cc9a4af6a6ee540cf11e4f1e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:49.439333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:49.440265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:49.440726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:49.485396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:49.488305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:49.624615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 818/1631] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:49.663962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:49.667338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:49.779955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf80dda-15e1-4a1e-83b2-db20f0e8cd1a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:49.785635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec8c6c-ddd4-4341-bce8-da7ba578eb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:49.786490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec8c6c-ddd4-4341-bce8-da7ba578eb67 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:49.786942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec8c6c-ddd4-4341-bce8-da7ba578eb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:49.787419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec8c6c-ddd4-4341-bce8-da7ba578eb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:49.803228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec8c6c-ddd4-4341-bce8-da7ba578eb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:49.804567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec8c6c-ddd4-4341-bce8-da7ba578eb67 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:49.819146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 814/1632] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:51.258001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.261665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:51.285384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7354e-171d-4940-a53c-158f700cdf4b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.292076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.294159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.294754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.295195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] PATH_INFO: `/v3/projects/1373d115e5ba401d96f2b60357e4d2b5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.296716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1373d115e5ba401d96f2b60357e4d2b5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.298050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:59:51.298050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Sep 10 20:59:51.298050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: Sep 10 20:59:51.299986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:51.466325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1373d115e5ba401d96f2b60357e4d2b5', 'request_id': 'req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.487777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1373d115e5ba401d96f2b60357e4d2b5', 'request_id': 'req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.500408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1373d115e5ba401d96f2b60357e4d2b5', 'request_id': 'req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.514234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:51.527617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] The token cache is being invalidate because project 1373d115e5ba401d96f2b60357e4d2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:59:51.528002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.530161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25e0c12f-64f5-4b0f-bac0-24c5fbccd1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.535648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 819/1633] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 20:59:51 2022] DELETE /identity/v3/projects/1373d115e5ba401d96f2b60357e4d2b5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:51.538234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.540525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.541657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.583113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:51.584591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:51.710057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 815/1634] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:51.719057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.719491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.719806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.744868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:51.746751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:51.760514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.771298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 820/1635] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:51.772621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1c900-16a0-4d56-99bb-39ff6b5e75a5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.779803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.780628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.781628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.781994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.782795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:51.783183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.785215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:51.805247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.812346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 816/1636] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:51.818372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d5c1e-93e9-4538-89c7-5629740c5098 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.824238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.825068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.825365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.825705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.827454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.829053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:51.873212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.887192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 821/1637] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/projects => generated 405 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:51.888580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b3d3-01fe-40fe-b995-d6e1696369dd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.914362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:51.973183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.974084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.974376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.974655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.975305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.978793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.009520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.010002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 817/1638] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:52.020971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70dc6e-64ed-4324-826a-e1ba555e77bd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.027189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.027900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.028351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.028598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.029350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.031099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.048691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.060460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 822/1639] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:52.062954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d2580-9a74-4b14-8d09-bb515a08f5ba admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.086650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c867d-2318-4a3e-9986-bdd81a28abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.087384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c867d-2318-4a3e-9986-bdd81a28abec admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.087886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c867d-2318-4a3e-9986-bdd81a28abec admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.088172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c867d-2318-4a3e-9986-bdd81a28abec admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.088885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c867d-2318-4a3e-9986-bdd81a28abec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.091978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c867d-2318-4a3e-9986-bdd81a28abec admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.124078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 818/1640] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:59:52.124812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.155632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24839b-cb77-48ca-a535-8a99c82998aa admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.160913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.163589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.163976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.164268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] PATH_INFO: `/v3/projects/daf5b93920984d38baa238022c0d464d/users/3fcc7105ebc54363b775ac19e953dcfe/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.185390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf5b93920984d38baa238022c0d464d, user_id=3fcc7105ebc54363b775ac19e953dcfe, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.189450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.211673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 823/1641] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:52 2022] PUT /identity/v3/projects/daf5b93920984d38baa238022c0d464d/users/3fcc7105ebc54363b775ac19e953dcfe/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:52.214092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79035431-1786-4e1c-8464-fdd0b4662bd1 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.214785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79035431-1786-4e1c-8464-fdd0b4662bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.215248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79035431-1786-4e1c-8464-fdd0b4662bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.239623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-79035431-1786-4e1c-8464-fdd0b4662bd1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:52.241192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-79035431-1786-4e1c-8464-fdd0b4662bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.380540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 819/1642] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:52.391761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.394890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.433049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e2d66-ad8d-4590-ae84-fe9a5ee2bba3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.448218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.449158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.449745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.450165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] PATH_INFO: `/v3/users/a0e373c5a6de4f82b29fcfae93e982cd` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.452199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e373c5a6de4f82b29fcfae93e982cd)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.454832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.626919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e373c5a6de4f82b29fcfae93e982cd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:52.660569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e373c5a6de4f82b29fcfae93e982cd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:52.661646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e373c5a6de4f82b29fcfae93e982cd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:52.666348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e373c5a6de4f82b29fcfae93e982cd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:52.678584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e373c5a6de4f82b29fcfae93e982cd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:52.727158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 824/1643] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:52 2022] DELETE /identity/v3/users/a0e373c5a6de4f82b29fcfae93e982cd => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:52.732933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.733436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.733696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.750235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.752864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.796140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:52.799202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.911810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4354-75b2-4ee6-ba9b-70695e0f58ce admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.922679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.923622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.923984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.924216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.928584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.942352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 820/1644] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:52.958283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.977037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:52.981339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899cc9a-e13b-40a4-a007-0fdd1222c16c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.991899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.992531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.992822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.993202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:53.013317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:53.015403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:53.029942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 821/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:53.110417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:53.114414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de0e54-f5eb-4a44-a572-122974865eb5 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:53.151832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 825/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:54.117946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:54.120848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:54.145633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403efe3-61aa-4820-85b7-75e3e3fe4c86 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:54.155684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:54.156492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.156773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.157514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] PATH_INFO: `/v3/projects/5704e83d2f7243c09ab2b1bb14fb03a5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.158713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5704e83d2f7243c09ab2b1bb14fb03a5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:54.165722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:54.200398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:54.244348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5704e83d2f7243c09ab2b1bb14fb03a5', 'request_id': 'req-3fdb884b-d3d8-4fd7-bbd5-87d359362163'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:54.260326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5704e83d2f7243c09ab2b1bb14fb03a5', 'request_id': 'req-3fdb884b-d3d8-4fd7-bbd5-87d359362163'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:54.276580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5704e83d2f7243c09ab2b1bb14fb03a5', 'request_id': 'req-3fdb884b-d3d8-4fd7-bbd5-87d359362163'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:54.291641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] The token cache is being invalidate because project 5704e83d2f7243c09ab2b1bb14fb03a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:59:54.291852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:54.292701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3fdb884b-d3d8-4fd7-bbd5-87d359362163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:54.298429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 822/1647] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 20:59:54 2022] DELETE /identity/v3/projects/5704e83d2f7243c09ab2b1bb14fb03a5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:54.300119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.300609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.300795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.336723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:54.338033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:54.445489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 826/1648] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:54.454101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.454101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.454364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.482914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:54.485148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:54.505490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 823/1649] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:54.510182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:54.524259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335f3b0-716f-46ec-846d-ed39d736e8cd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:54.532351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:54.533258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.533692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.534122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.535053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:54.535497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:54.537405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:54.559463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:54.559654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 827/1650] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:54.582284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc36f5c-380b-43bd-8b74-9610b559f9a6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:54.597816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:54.599163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.599163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.599163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.599367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:54.601398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:54.668136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:54.677243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 824/1651] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:54 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:54.688938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab239-e0ee-46b1-a1be-d6e2b49badb4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:54.859286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:54.860265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.860900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.862117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.863669 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:54.866692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:54.936859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:54.954408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 828/1652] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:54 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:54.971697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a0fcf-4285-419f-b275-0c350a8d3533 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:54.990950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:54.991594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:54.991873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:54.992147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:54.995441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:55.002131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:55.073972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 825/1653] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:55.078072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:55.110032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43842f-c70a-43dc-a741-cd91e7fea889 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:55.126117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b3229-2001-4f9c-91d0-f2f26df5beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:55.127109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b3229-2001-4f9c-91d0-f2f26df5beda admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.127464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b3229-2001-4f9c-91d0-f2f26df5beda admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.128232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b3229-2001-4f9c-91d0-f2f26df5beda admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.131351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b3229-2001-4f9c-91d0-f2f26df5beda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:55.132812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b3229-2001-4f9c-91d0-f2f26df5beda admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:55.177372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 829/1654] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:55.180456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:55.246245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303009c-c505-4a3e-962b-f975606f08c1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:55.275589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:55.276292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.276577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.276857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] PATH_INFO: `/v3/projects/296b81823f3548a4831962e853f0dc36/users/d57f7b28dcbd4bb0a2c16aaf4cc6e48a/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.342110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296b81823f3548a4831962e853f0dc36, user_id=d57f7b28dcbd4bb0a2c16aaf4cc6e48a, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:55.351091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:55.394709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 826/1655] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:55 2022] PUT /identity/v3/projects/296b81823f3548a4831962e853f0dc36/users/d57f7b28dcbd4bb0a2c16aaf4cc6e48a/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:55.405536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc7cfb-e2da-401e-8779-daf220c52406 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.405536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc7cfb-e2da-401e-8779-daf220c52406 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.405536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc7cfb-e2da-401e-8779-daf220c52406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.547683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9dcc7cfb-e2da-401e-8779-daf220c52406 None None] MFA Rules not processed for user `d57f7b28dcbd4bb0a2c16aaf4cc6e48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:55.548858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9dcc7cfb-e2da-401e-8779-daf220c52406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:55.569900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:55.729357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 830/1656] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:55.730634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:55.822195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.823058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.823486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.923218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] MFA Rules not processed for user `3fcc7105ebc54363b775ac19e953dcfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:55.924458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:55.952290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffb45d-0b32-409b-8a98-9551c3f493b2 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:55.967668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:55.968665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.969148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.969779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.970610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:55.986380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:56.062817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 827/1657] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:55 2022] POST /identity/v3/projects => generated 365 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:56.064948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.196115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:56.205381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 831/1658] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:56.213341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.218024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:56.285225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9399a3-4a04-4df7-8fd5-259fd8997801 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.317962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903d9e-7bf4-492e-98ba-8f46cddd1c9b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.339712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:56.339712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:56.340047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:56.340255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:56.347333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:56.352175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:56.426875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 828/1659] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:56 2022] POST /identity/v3/users => generated 289 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:56.432036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.476214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92917e6-65b5-464d-849a-b8b33f9844f4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.496278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:56.507508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:56.507809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:56.508082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:56.509265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:56.518900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:56.519552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:56.519821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:56.520099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:56.520747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:56.521338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:56.522704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:56.583058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 829/1660] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:56.589827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.610351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 832/1661] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:56 2022] POST /identity/v3/projects => generated 405 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:56.611466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.630978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33604-4634-4f3e-9c09-6f9a0c6c5b9b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.657324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:56.740659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e2720-aff0-40eb-b912-00949f94df85 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.767685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:56.768530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:56.768805 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:56.769085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:56.770111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:56.773454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:56.843544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 833/1662] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:56 2022] POST /identity/v3/users => generated 309 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:56.844851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.875520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809de20-68ea-498d-b056-ddf0a4a79e91 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.901978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:56.903142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:56.903448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:56.903754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:56.904664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:56.907754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:56.919853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:56.921179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:56.921489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:56.928775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] PATH_INFO: `/v3/projects/241d547799034d1fa422901cd5d6d4ef/users/396a88a2e4d549218cdfeb7e7a577453/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:56.935867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 834/1663] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:56.940559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:56.984062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11608-6d97-4a53-a2db-c9f027f95bf1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:56.999934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241d547799034d1fa422901cd5d6d4ef, user_id=396a88a2e4d549218cdfeb7e7a577453, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.002861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.007742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.011556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.013097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.013745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] PATH_INFO: `/v3/projects/f84bbbe3ee9b4542b76b2580098ab655/users/50991aafa169469da59b87b69d96a950/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.036533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 830/1664] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:56 2022] PUT /identity/v3/projects/241d547799034d1fa422901cd5d6d4ef/users/396a88a2e4d549218cdfeb7e7a577453/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:57.041062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.072534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84bbbe3ee9b4542b76b2580098ab655, user_id=50991aafa169469da59b87b69d96a950, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.075915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.113336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 835/1665] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:56 2022] PUT /identity/v3/projects/f84bbbe3ee9b4542b76b2580098ab655/users/50991aafa169469da59b87b69d96a950/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:57.114800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.214869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150c91-bed4-4c46-953e-0f1482d957b1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.322726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c007e9-57e6-4173-b5e2-32b607c95bab admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.331767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.333223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.339352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.339714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.340427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.342133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.342698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:57.376966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 836/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:57.377970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.394661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.396242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.398947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.399593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.401152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.404155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.411344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a8dbc-fae2-4306-9c8a-940e0983a20c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.416798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b796377-0a06-49f3-aa64-b0bd7f4037a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.417596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b796377-0a06-49f3-aa64-b0bd7f4037a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.417897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b796377-0a06-49f3-aa64-b0bd7f4037a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.418181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b796377-0a06-49f3-aa64-b0bd7f4037a9 admin admin] PATH_INFO: `/v3/projects/f84bbbe3ee9b4542b76b2580098ab655/users/50991aafa169469da59b87b69d96a950/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.425920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b796377-0a06-49f3-aa64-b0bd7f4037a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84bbbe3ee9b4542b76b2580098ab655, user_id=50991aafa169469da59b87b69d96a950, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.428891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b796377-0a06-49f3-aa64-b0bd7f4037a9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.437650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 831/1667] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:57.438724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.468969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 837/1668] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:57 2022] PUT /identity/v3/projects/f84bbbe3ee9b4542b76b2580098ab655/users/50991aafa169469da59b87b69d96a950/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:57.484293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f121f-501a-4ea9-a1af-078ecaaa7f99 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.610562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.611315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.616362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.616988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] PATH_INFO: `/v3/projects/241d547799034d1fa422901cd5d6d4ef/users/396a88a2e4d549218cdfeb7e7a577453/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.629156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241d547799034d1fa422901cd5d6d4ef, user_id=396a88a2e4d549218cdfeb7e7a577453, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.635836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.663291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 832/1669] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:57 2022] PUT /identity/v3/projects/241d547799034d1fa422901cd5d6d4ef/users/396a88a2e4d549218cdfeb7e7a577453/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:57.675295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.676602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.677041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.735420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] MFA Rules not processed for user `396a88a2e4d549218cdfeb7e7a577453`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:57.736926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:57.803870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.806192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:57.887072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:57.955331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 838/1670] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:57.961342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.966478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:58.010503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce339f-f0bc-4842-bd78-80a179ef7419 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:58.023500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:58.027683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:58.028066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:58.028358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] PATH_INFO: `/v3/users/33a6c442d9f34521b7594a476741bfdd` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:58.035365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a6c442d9f34521b7594a476741bfdd)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:58.037494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:58.167902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f387ae-94a4-4c1c-8481-ac1ba2d1f8d0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:58.173784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a6c442d9f34521b7594a476741bfdd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:58.178156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556e621-bce9-410e-8c2c-0786eeeb086b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:58.178797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556e621-bce9-410e-8c2c-0786eeeb086b service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:58.179080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556e621-bce9-410e-8c2c-0786eeeb086b service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:58.179359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556e621-bce9-410e-8c2c-0786eeeb086b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:58.201683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a6c442d9f34521b7594a476741bfdd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:58.202322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a6c442d9f34521b7594a476741bfdd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:58.208666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556e621-bce9-410e-8c2c-0786eeeb086b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:58.211823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556e621-bce9-410e-8c2c-0786eeeb086b service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:58.219921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a6c442d9f34521b7594a476741bfdd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:58.230359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:58.239041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 839/1671] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:58.246402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a6c442d9f34521b7594a476741bfdd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:58.265817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:58.266109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:58.266527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:58.273821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 833/1672] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:57 2022] DELETE /identity/v3/users/33a6c442d9f34521b7594a476741bfdd => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:59:58.295938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:58.298609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:58.455568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 840/1673] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:58.478897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:58.480681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:58.676994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221595-ae50-4fc8-b370-ef26fddca40c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:58.687297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9ffe-5029-406c-aa17-d1fb2594a2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:58.688189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9ffe-5029-406c-aa17-d1fb2594a2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:58.688313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9ffe-5029-406c-aa17-d1fb2594a2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:58.688502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9ffe-5029-406c-aa17-d1fb2594a2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:58.723093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb9ffe-5029-406c-aa17-d1fb2594a2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:58.724575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb9ffe-5029-406c-aa17-d1fb2594a2b3 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:58.751194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 834/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:59.352560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:59.353701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:59.388288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2849d50-4c64-4dcf-bb0c-7b960af46d0f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:59.398127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:59.405126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:59.406031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:59.406031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] PATH_INFO: `/v3/users/d57f7b28dcbd4bb0a2c16aaf4cc6e48a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:59.413812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57f7b28dcbd4bb0a2c16aaf4cc6e48a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:59.416696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:59.584761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57f7b28dcbd4bb0a2c16aaf4cc6e48a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:59.607040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57f7b28dcbd4bb0a2c16aaf4cc6e48a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:59.608257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57f7b28dcbd4bb0a2c16aaf4cc6e48a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:59.609254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57f7b28dcbd4bb0a2c16aaf4cc6e48a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:59.628307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57f7b28dcbd4bb0a2c16aaf4cc6e48a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:59.661236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:59.675239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 841/1675] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:59 2022] DELETE /identity/v3/users/d57f7b28dcbd4bb0a2c16aaf4cc6e48a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:59.679773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028e5a3-844f-47b5-8e2f-19561b9e2085 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:59.680627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028e5a3-844f-47b5-8e2f-19561b9e2085 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:59.680627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028e5a3-844f-47b5-8e2f-19561b9e2085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:59.717210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c028e5a3-844f-47b5-8e2f-19561b9e2085 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:59.719696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c028e5a3-844f-47b5-8e2f-19561b9e2085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:59.866101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 835/1676] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 20:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:59.891189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:59.894179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:00.063650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974f2cf-138c-4cde-bcc4-aa2e9a3ce23f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:00.076814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f522be-0b5d-48d1-80f7-6321fdff0824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:00.079108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f522be-0b5d-48d1-80f7-6321fdff0824 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:00.079477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f522be-0b5d-48d1-80f7-6321fdff0824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:00.081255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f522be-0b5d-48d1-80f7-6321fdff0824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:00.102384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f522be-0b5d-48d1-80f7-6321fdff0824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:00.109617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f522be-0b5d-48d1-80f7-6321fdff0824 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:00.140919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 842/1677] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:01.267387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f50ae-2237-4ebd-9277-cb95b58c3066 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:01.267858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f50ae-2237-4ebd-9277-cb95b58c3066 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:01.267941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f50ae-2237-4ebd-9277-cb95b58c3066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:01.323519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-474f50ae-2237-4ebd-9277-cb95b58c3066 None None] MFA Rules not processed for user `50991aafa169469da59b87b69d96a950`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:01.325321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-474f50ae-2237-4ebd-9277-cb95b58c3066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:01.573568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 836/1678] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:01.673111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:01.674588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:01.676994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:01.759653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] MFA Rules not processed for user `12f441d491f846018630b0871734aab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:01.762064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:01.971471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 843/1679] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:01.986767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:01.986767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:01.986767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:02.049684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] MFA Rules not processed for user `12f441d491f846018630b0871734aab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:02.054045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:02.110906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 837/1680] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:02.120536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:02.122080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:02.329106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361d0a-dd4b-491f-a839-ecf2dfcf639a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:02.334785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:02.338366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:02.338651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:02.338928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:02.353247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:02.354948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:02.383620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 844/1681] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:02.390249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:02.392071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:02.406388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc32ace-1956-4eef-ac86-c85ad1df5286 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:02.412327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:02.412980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:02.413261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:02.413642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] PATH_INFO: `/v3/projects/296b81823f3548a4831962e853f0dc36` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:02.415597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296b81823f3548a4831962e853f0dc36)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:02.417581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:02.428892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:02.505969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296b81823f3548a4831962e853f0dc36', 'request_id': 'req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.516489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296b81823f3548a4831962e853f0dc36', 'request_id': 'req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.540680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296b81823f3548a4831962e853f0dc36', 'request_id': 'req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.572560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] The token cache is being invalidate because project 296b81823f3548a4831962e853f0dc36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:02.577889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.581951 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.595765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 838/1682] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:02 2022] DELETE /identity/v3/projects/296b81823f3548a4831962e853f0dc36 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:02.599338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:02.618319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a95fbd-dc79-4590-8bbd-94a0ed79f228 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:02.640309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:02.687739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:02.689912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:02.750582 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:02.753205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:02.754574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:02.756875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] PATH_INFO: `/v3/users/396a88a2e4d549218cdfeb7e7a577453` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:02.761004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] RBAC: Authorizing `identity:delete_user(user_id=396a88a2e4d549218cdfeb7e7a577453)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:02.763443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:02.856614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c4c40-ce80-4d1d-9595-52cf91a76bf6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:02.857234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396a88a2e4d549218cdfeb7e7a577453'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.874316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:02.874994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:02.875280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:02.875560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] PATH_INFO: `/v3/projects/479ec0254e1a4edeb48bda9a262981b3` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:02.880048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479ec0254e1a4edeb48bda9a262981b3)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:02.881703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396a88a2e4d549218cdfeb7e7a577453'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.882873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:02.883861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396a88a2e4d549218cdfeb7e7a577453'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.885053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396a88a2e4d549218cdfeb7e7a577453'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.895189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-408e8b44-36af-43aa-a3a1-5d945f80226e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396a88a2e4d549218cdfeb7e7a577453'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:02.949352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 845/1683] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:02 2022] DELETE /identity/v3/users/396a88a2e4d549218cdfeb7e7a577453 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:03.023105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479ec0254e1a4edeb48bda9a262981b3', 'request_id': 'req-604ec7c6-1f26-40f9-be4f-0152ee2fda17'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:03.038150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479ec0254e1a4edeb48bda9a262981b3', 'request_id': 'req-604ec7c6-1f26-40f9-be4f-0152ee2fda17'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:03.061316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479ec0254e1a4edeb48bda9a262981b3', 'request_id': 'req-604ec7c6-1f26-40f9-be4f-0152ee2fda17'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:03.109720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] The token cache is being invalidate because project 479ec0254e1a4edeb48bda9a262981b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:03.110423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:03.114381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-604ec7c6-1f26-40f9-be4f-0152ee2fda17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:03.120849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 839/1684] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:02 2022] DELETE /identity/v3/projects/479ec0254e1a4edeb48bda9a262981b3 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:03.128898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:03.129365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:03.130027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:03.206343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:03.210794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:03.393862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 846/1685] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:03.402552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:03.403080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:03.404821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:03.453537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:03.456835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:03.490822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 840/1686] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:03.492085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:03.508195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc37f4c-0856-464b-92c8-ac81d50864f7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:03.519428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:03.520111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:03.520399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:03.520678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:03.521360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:03.522222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:03.528349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:03.562226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 847/1687] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:03.566595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:03.578782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6d24-7150-4e0c-98ab-41892735979f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:03.597316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:03.597982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:03.598265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:03.598539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:03.599192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:03.601146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:03.642292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:03.672135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 841/1688] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:03 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:03.676126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:03.695170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea2194-a0a2-4aef-b79f-7b74db282615 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:03.724969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:03.813523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:03.814177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:03.814463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:03.814738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:03.815550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:03.817631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:03.872352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 848/1689] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:03 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:03.873542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:03.913311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba4de-ccf9-4274-beab-1a3b7274fe99 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:03.936316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:03.937221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:03.937879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:03.938389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:03.939320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:03.941505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:03.966625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 842/1690] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:00:03.971567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:03.996056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61b8b-bf3a-4ec1-b7d9-546ea2279743 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:04.010743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf7f3a-38a9-4956-a742-7fd48602a9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:04.011873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7f3a-38a9-4956-a742-7fd48602a9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.012158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7f3a-38a9-4956-a742-7fd48602a9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.012575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7f3a-38a9-4956-a742-7fd48602a9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.013849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf7f3a-38a9-4956-a742-7fd48602a9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:04.015462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf7f3a-38a9-4956-a742-7fd48602a9d8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:04.047525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 849/1691] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:04.051711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:04.072707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a45c68-5c83-4cfb-8f64-7aa7f5e225c8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:04.090856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:04.091769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.092112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.092506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] PATH_INFO: `/v3/projects/e461091b85754b7db5d8f667b14d8373/users/ec0bc5c09ef4418ea01915e68993f31d/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.129562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e461091b85754b7db5d8f667b14d8373, user_id=ec0bc5c09ef4418ea01915e68993f31d, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:04.133279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:04.173836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 843/1692] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:04 2022] PUT /identity/v3/projects/e461091b85754b7db5d8f667b14d8373/users/ec0bc5c09ef4418ea01915e68993f31d/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:04.175688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f315654-a1d5-4f30-b9c7-f79511d68b21 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.175965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f315654-a1d5-4f30-b9c7-f79511d68b21 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.176828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f315654-a1d5-4f30-b9c7-f79511d68b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.242251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0f315654-a1d5-4f30-b9c7-f79511d68b21 None None] MFA Rules not processed for user `ec0bc5c09ef4418ea01915e68993f31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:04.244403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0f315654-a1d5-4f30-b9c7-f79511d68b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:04.434873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 850/1693] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:04.436246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:04.451672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea5966-b742-41a2-b11f-8655911e45c4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.451961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea5966-b742-41a2-b11f-8655911e45c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.452855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea5966-b742-41a2-b11f-8655911e45c4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.460798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 851/1694] 10.222.0.23 () {54 vars in 912 bytes} [Sat Sep 10 21:00:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:04.468673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.469221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.469747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.656838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] MFA Rules not processed for user `934be918dc8c4fa59988e5bb22e56eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:04.662151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:04.715396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af080b38-ed32-4499-a299-2151816fcc3e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:04.737180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae9835-9f53-42d4-9fc3-ef835008723f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:04.737180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae9835-9f53-42d4-9fc3-ef835008723f admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.737180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae9835-9f53-42d4-9fc3-ef835008723f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.744249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae9835-9f53-42d4-9fc3-ef835008723f admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.746461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae9835-9f53-42d4-9fc3-ef835008723f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:04.753306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae9835-9f53-42d4-9fc3-ef835008723f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:04.869093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 844/1695] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:04 2022] POST /identity/v3/projects => generated 365 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:04.870978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:04.872333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:04.872875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:04.915355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 852/1696] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:00:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:04.919363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:04.922982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:04.935379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] MFA Rules not processed for user `47f9883392364d3fb5d0b921d6f451d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:04.936912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.006310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce10c21-662f-4c04-9ccf-3c0afad1441b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:05.105893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:05.198947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:05.202458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.202747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.203028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] PATH_INFO: `/v3/projects/241d547799034d1fa422901cd5d6d4ef` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:05.223927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 845/1697] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:05.224437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241d547799034d1fa422901cd5d6d4ef)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:05.224437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:05.233350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:05.265105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b4415-223d-45bd-8dd6-79916a446865 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:05.284842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:05.285505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.286050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.286050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:05.294824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:05.300080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:05.343120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241d547799034d1fa422901cd5d6d4ef', 'request_id': 'req-f56e537e-f905-4535-9070-7269643254e8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:05.361324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 846/1698] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:05 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:05.361536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241d547799034d1fa422901cd5d6d4ef', 'request_id': 'req-f56e537e-f905-4535-9070-7269643254e8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:05.361636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:05.367101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.385902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241d547799034d1fa422901cd5d6d4ef', 'request_id': 'req-f56e537e-f905-4535-9070-7269643254e8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:05.403264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] The token cache is being invalidate because project 241d547799034d1fa422901cd5d6d4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:05.403557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:05.405844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f56e537e-f905-4535-9070-7269643254e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:05.409591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 853/1699] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:04 2022] DELETE /identity/v3/projects/241d547799034d1fa422901cd5d6d4ef => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:05.411249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.411535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.414765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:05.443173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:05.456401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] MFA Rules not processed for user `47f9883392364d3fb5d0b921d6f451d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:05.457813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.563776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ca354-4786-4343-b24d-1525983155d5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:05.582308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 854/1700] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:05.582649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:05.590084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.737313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:05.754865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db053c3a-dc0d-459c-bc48-aac1a13facd7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:05.758941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.760062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.760464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:05.764875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97166aef-a3a7-46ca-989b-817b065e8585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:05.765974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166aef-a3a7-46ca-989b-817b065e8585 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.766514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166aef-a3a7-46ca-989b-817b065e8585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.767067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166aef-a3a7-46ca-989b-817b065e8585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:05.767838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166aef-a3a7-46ca-989b-817b065e8585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:05.769770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166aef-a3a7-46ca-989b-817b065e8585 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:05.770322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.780533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:05.785517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 855/1701] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:05.788079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.790585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.791486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:05.802945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:05.805178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:05.819038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 847/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:05.820362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:05.822129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.926175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:05.932279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:05.945989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 856/1703] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:05.947185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:05.962256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23974d38-b92e-4a02-a91f-539e69c38dd7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:05.981210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f326d45-ddff-460e-ad23-21285f15c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:05.982223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f326d45-ddff-460e-ad23-21285f15c873 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:05.982640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f326d45-ddff-460e-ad23-21285f15c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:05.983320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f326d45-ddff-460e-ad23-21285f15c873 admin admin] PATH_INFO: `/v3/projects/e977c936189a47d0b77fd4e9a0eef203/users/2ea37598f67e4842ad08ae819e0af322/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.035487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f326d45-ddff-460e-ad23-21285f15c873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e977c936189a47d0b77fd4e9a0eef203, user_id=2ea37598f67e4842ad08ae819e0af322, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:06.043037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f326d45-ddff-460e-ad23-21285f15c873 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:06.085681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c989-c802-4f0d-a05d-ccddc41d81c5 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:06.111377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 857/1704] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:05 2022] PUT /identity/v3/projects/e977c936189a47d0b77fd4e9a0eef203/users/2ea37598f67e4842ad08ae819e0af322/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:06.122240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301b969-9d02-438d-b29c-3df6f5a06e86 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.122601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301b969-9d02-438d-b29c-3df6f5a06e86 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.122804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301b969-9d02-438d-b29c-3df6f5a06e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.212181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a301b969-9d02-438d-b29c-3df6f5a06e86 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:06.216811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a301b969-9d02-438d-b29c-3df6f5a06e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:06.312467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:06.313416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.313564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.314817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.319428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:06.358089 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:06.385873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 858/1705] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:06.385873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ca68-c22e-41c1-8bf8-983928d40e87 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.385873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ca68-c22e-41c1-8bf8-983928d40e87 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.386203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ca68-c22e-41c1-8bf8-983928d40e87 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.387644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 859/1706] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:06.399906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ca68-c22e-41c1-8bf8-983928d40e87 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:06.430925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ca68-c22e-41c1-8bf8-983928d40e87 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:06.441684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:06.442496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.442941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.444195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.445031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:06.449612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:06.478915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 860/1707] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:06.481506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:06.498842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:06.500733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:06.503832 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b14c1-23c8-43df-be57-3c0f08354345 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:06.519527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:06.520433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.521014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.521435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.522530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:06.523246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:06.529543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 848/1708] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:06.530405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:06.542154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532967e0-2b07-41ac-8001-d3d1e92b9f6d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:06.556354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cebfd8-03df-4553-921d-044db3132a5d service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:06.569242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:06.570003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.570312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.570595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] PATH_INFO: `/v3/projects/e977c936189a47d0b77fd4e9a0eef203/users/2ea37598f67e4842ad08ae819e0af322/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.575237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e977c936189a47d0b77fd4e9a0eef203, user_id=2ea37598f67e4842ad08ae819e0af322, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:06.580519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:06.618386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 861/1710] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:06.620205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce5770-b5ef-4487-b793-dff2b66e784d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.622059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 849/1710] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:06 2022] PUT /identity/v3/projects/e977c936189a47d0b77fd4e9a0eef203/users/2ea37598f67e4842ad08ae819e0af322/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:06.623062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:06.625915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce5770-b5ef-4487-b793-dff2b66e784d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.628120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce5770-b5ef-4487-b793-dff2b66e784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.718162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d6ce5770-b5ef-4487-b793-dff2b66e784d None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:06.719394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d6ce5770-b5ef-4487-b793-dff2b66e784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:06.848021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bc7d-ab44-4d8a-9cb6-abe8c801694b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:06.860714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:06.861417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.861732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.862024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.862680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:06.870914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:06.921235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 862/1711] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:00:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:06.926740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:06.928583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:06.928954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:06.960033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 850/1712] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:06 2022] POST /identity/v3/projects => generated 367 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:06.962129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.000578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] MFA Rules not processed for user `2ea37598f67e4842ad08ae819e0af322`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:07.002068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:07.093182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:07.108070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ba592-defb-48a4-8a05-4c8973587b2d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:07.122687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:07.123629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.124015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.124548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.125315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:07.131347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:07.194165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 851/1713] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:06 2022] POST /identity/v3/users => generated 284 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:07.199147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.219026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 863/1714] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:07.234218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969c30-3c60-490a-86d1-286acc9d43c8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:07.240483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.244873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:07.248007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:07.248828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.249861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.250513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.251125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:07.253591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:07.293357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 852/1715] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:07.294388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.298654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:07.318306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:07.396405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2842b-68ec-4dad-a3e8-8354684c824e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:07.416900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:07.417730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.418911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.419194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.462088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:07.470070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:07.497325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 864/1716] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:07.501051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.542020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5f4c3-586e-4a71-a8ac-0a000b64f5dc service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:07.542970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6bb69-ce0d-4a67-ae52-f4a4220f8850 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:07.561992 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b07449-9dc5-45b9-9cb8-0b09be43421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:07.562705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b07449-9dc5-45b9-9cb8-0b09be43421d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.562898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b07449-9dc5-45b9-9cb8-0b09be43421d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.563793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b07449-9dc5-45b9-9cb8-0b09be43421d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.564495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:07.565017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.565288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.567403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.568308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b07449-9dc5-45b9-9cb8-0b09be43421d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:07.574141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b07449-9dc5-45b9-9cb8-0b09be43421d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:07.607494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:07.609374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:07.614901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 865/1717] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:07.616583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680a187-142a-4d4e-b971-9dc0cd9325c6 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.616706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680a187-142a-4d4e-b971-9dc0cd9325c6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.616958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680a187-142a-4d4e-b971-9dc0cd9325c6 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.630252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 866/1718] 10.222.0.23 () {54 vars in 961 bytes} [Sat Sep 10 21:00:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:07.633506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.634969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.634969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.661730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 853/1719] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:07.663222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.690341 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08c73-747d-4447-8a19-9c83f547b260 service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:07.711610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ae75-5116-4625-a3d9-e81aa76f8400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:07.713029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ae75-5116-4625-a3d9-e81aa76f8400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.716504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ae75-5116-4625-a3d9-e81aa76f8400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.716800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ae75-5116-4625-a3d9-e81aa76f8400 admin admin] PATH_INFO: `/v3/projects/d02039664fe44d1182f931c7f7a6cd02/users/52bd3e0e95d6434e8c120113e5f7f226/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.729825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] MFA Rules not processed for user `16610ae2c3214396b78fb1caff01ab6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:07.732386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:07.766410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ae75-5116-4625-a3d9-e81aa76f8400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d02039664fe44d1182f931c7f7a6cd02, user_id=52bd3e0e95d6434e8c120113e5f7f226, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:07.772467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ae75-5116-4625-a3d9-e81aa76f8400 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:07.807512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 854/1720] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:07 2022] PUT /identity/v3/projects/d02039664fe44d1182f931c7f7a6cd02/users/52bd3e0e95d6434e8c120113e5f7f226/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:07.809243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983550a-8a7a-44f8-bff5-74895c180d46 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:07.809561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983550a-8a7a-44f8-bff5-74895c180d46 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:07.809847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983550a-8a7a-44f8-bff5-74895c180d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:07.860039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7983550a-8a7a-44f8-bff5-74895c180d46 None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:07.864219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7983550a-8a7a-44f8-bff5-74895c180d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:07.933631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 867/1721] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:07.933631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:07.948585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91822262-8de3-49fa-8b13-5b4657a3452e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:08.035621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 855/1722] 10.222.0.23 () {58 vars in 1058 bytes} [Sat Sep 10 21:00:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:08.041580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.041888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.042175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.066760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:08.073989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:08.074668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.074952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.075225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.078769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:08.139290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] MFA Rules not processed for user `a22c6bd0fac8443c844d7734aba97ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:08.141202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:08.224467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:08.226459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a363-5c40-4b95-a325-2410bbe39714 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:08.255686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 868/1723] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:08.258609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.259083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.259545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.318771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] MFA Rules not processed for user `52bd3e0e95d6434e8c120113e5f7f226`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:08.320248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:08.339871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:08.370491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 856/1724] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:08.372708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:08.381222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:08.531643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 869/1725] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:08.535925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:08.585348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6aaa-945b-4e28-9937-17f847dc67a3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:08.609403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:08.609776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.610054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.610332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.619662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:08.649939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:08.652703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3a67c-612e-4585-a36b-1ce1098bc002 service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:08.658735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155403c9-5b12-4d99-9d1a-b412f8526d8e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:08.676758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7f3c-7352-4367-bcc0-738245612f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:08.678313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7f3c-7352-4367-bcc0-738245612f7a service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.678474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7f3c-7352-4367-bcc0-738245612f7a service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.681134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7f3c-7352-4367-bcc0-738245612f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.685562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 870/1726] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:08.688698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f0081-aea5-4e39-a5e5-3d7d00da3d42 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.688996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f0081-aea5-4e39-a5e5-3d7d00da3d42 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.689283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f0081-aea5-4e39-a5e5-3d7d00da3d42 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.692940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 871/1727] 10.222.0.23 () {54 vars in 912 bytes} [Sat Sep 10 21:00:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:08.697106 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f0081-aea5-4e39-a5e5-3d7d00da3d42 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:08.874458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f0081-aea5-4e39-a5e5-3d7d00da3d42 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:08.879925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910476c-cc22-4251-8527-093a9732aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:08.880704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910476c-cc22-4251-8527-093a9732aeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.881214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910476c-cc22-4251-8527-093a9732aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.881708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910476c-cc22-4251-8527-093a9732aeb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.882886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910476c-cc22-4251-8527-093a9732aeb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:08.885077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910476c-cc22-4251-8527-093a9732aeb3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:08.925505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7f3c-7352-4367-bcc0-738245612f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:08.933426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 872/1728] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:08 2022] POST /identity/v3/projects => generated 367 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:08.936887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7f3c-7352-4367-bcc0-738245612f7a service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:08.938839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.939292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.939722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.955918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 857/1729] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:08.963360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:08.964012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:08.964448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:08.979012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:09.017532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] MFA Rules not processed for user `934be918dc8c4fa59988e5bb22e56eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:09.019287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:09.026204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] MFA Rules not processed for user `6e24c7d558f74ce181aad463bf18ee52`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:09.027768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:09.212874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 873/1730] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:09.231620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:09.294203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 858/1731] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:09.296667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:09.410719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:09.472916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067de9ec-c221-4a55-b4d5-f5d42f858761 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:09.479164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:09.480056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.480830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.481286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.482136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:09.484526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:09.532773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 874/1732] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:09 2022] POST /identity/v3/users => generated 290 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:09.533979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:09.536064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:09.554523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc7875-3d02-4997-99a4-ec94ff4b6299 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:09.562689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:09.563623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.563911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.564184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.579264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:09.581437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:09.600748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 875/1733] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:09.606603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:09.619589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b307e44-bfc1-466f-a6d3-ea455fe65966 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:09.619790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b2f4f-68e5-4557-a448-4ac03e130e83 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:09.625359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd91f9-4b18-4677-85e3-50c7af8ffbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:09.626557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd91f9-4b18-4677-85e3-50c7af8ffbeb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.628170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd91f9-4b18-4677-85e3-50c7af8ffbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.628660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd91f9-4b18-4677-85e3-50c7af8ffbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.629390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd91f9-4b18-4677-85e3-50c7af8ffbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:09.634538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd91f9-4b18-4677-85e3-50c7af8ffbeb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:09.638493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:09.639266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.639664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.640687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.665713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 876/1734] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:09.669499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a7d9a-a082-4b55-af70-176ad362a96f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.669793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a7d9a-a082-4b55-af70-176ad362a96f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.670142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a7d9a-a082-4b55-af70-176ad362a96f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.676909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 877/1735] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:00:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:09.676909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a7d9a-a082-4b55-af70-176ad362a96f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:09.729280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a7d9a-a082-4b55-af70-176ad362a96f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:09.735242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb363246-eb4e-410a-ab2c-33f1ecc75b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:09.736029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb363246-eb4e-410a-ab2c-33f1ecc75b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.736411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb363246-eb4e-410a-ab2c-33f1ecc75b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.737525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb363246-eb4e-410a-ab2c-33f1ecc75b1f admin admin] PATH_INFO: `/v3/projects/d4f1a2bc9a814824917f8402c02f490d/users/588e9ab78e074d028cb1849883f18d10/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.766800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb363246-eb4e-410a-ab2c-33f1ecc75b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f1a2bc9a814824917f8402c02f490d, user_id=588e9ab78e074d028cb1849883f18d10, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:09.770927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb363246-eb4e-410a-ab2c-33f1ecc75b1f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:09.815849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 878/1736] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:09 2022] PUT /identity/v3/projects/d4f1a2bc9a814824917f8402c02f490d/users/588e9ab78e074d028cb1849883f18d10/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:09.829626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.833904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.834224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.899777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] MFA Rules not processed for user `df16503b8f2e4d9bb3cff4275a39d994`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:09.909569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:10.004176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:10.017946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:10.064381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:10.073044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 859/1737] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:10.074347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:10.096489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 879/1738] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:10.098810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:10.099674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b89ad6-f0c7-401e-ad8f-639f669caafd service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:10.100852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:10.360242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:10.362593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:10.363344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:10.364532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:10.366683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:10.373120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:10.422021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e7d2f-7df3-47d5-88a3-d545dabdf1cf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:10.436472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 860/1739] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:10.439042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:10.446271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:10.452153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:10.452538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:10.453900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] PATH_INFO: `/v3/users/3fcc7105ebc54363b775ac19e953dcfe` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:10.459129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fcc7105ebc54363b775ac19e953dcfe)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:10.465871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:10.490643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c8b95-d013-4959-b678-51d83f875a7d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:10.512124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:10.515013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:10.515300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:10.515580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:10.558345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:10.562989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:10.599904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 861/1740] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:10.608789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:10.660535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03322f-d261-4ab2-89b0-3186e6a03bfe service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:10.673296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:10.674443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:10.675206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:10.675626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fcc7105ebc54363b775ac19e953dcfe'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:10.677544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] PATH_INFO: `/v3/projects/d4f1a2bc9a814824917f8402c02f490d/users/588e9ab78e074d028cb1849883f18d10/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:10.682558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f1a2bc9a814824917f8402c02f490d, user_id=588e9ab78e074d028cb1849883f18d10, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:10.690739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:10.705659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fcc7105ebc54363b775ac19e953dcfe'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:10.707698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fcc7105ebc54363b775ac19e953dcfe'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:10.708999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fcc7105ebc54363b775ac19e953dcfe'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:10.717110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 862/1741] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:10 2022] PUT /identity/v3/projects/d4f1a2bc9a814824917f8402c02f490d/users/588e9ab78e074d028cb1849883f18d10/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:10.718233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:10.727662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:10.745224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4f0cd526-d91a-4c2e-824b-9e9ebb80c311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fcc7105ebc54363b775ac19e953dcfe'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:10.820227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:10.829234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 880/1742] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:10 2022] DELETE /identity/v3/users/3fcc7105ebc54363b775ac19e953dcfe => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:10.840095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5313f1-4174-4561-b723-f78e46121e22 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:10.840919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5313f1-4174-4561-b723-f78e46121e22 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:10.842520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5313f1-4174-4561-b723-f78e46121e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:11.030364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6e5313f1-4174-4561-b723-f78e46121e22 None None] MFA Rules not processed for user `588e9ab78e074d028cb1849883f18d10`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:11.042696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6e5313f1-4174-4561-b723-f78e46121e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:11.153012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1471aa-2079-4fe9-9e65-2fa04e7c08c1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:11.484974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:11.519584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 881/1743] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 686 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:11.525212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:11.525212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:11.528079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:11.592578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:11.597039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:11.597437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:11.597852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:11.609776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:11.611925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] MFA Rules not processed for user `47f9883392364d3fb5d0b921d6f451d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:11.614904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:11.907336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:11.912682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:11.922569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 882/1744] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:11.927043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:11.935247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:11.970693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 863/1745] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 1252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:12.167272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6022c4-5204-4342-b847-f1fdb85a5d21 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:12.171531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:12.172222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:12.172502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:12.172772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:12.196574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:12.200536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:12.278188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 883/1746] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:12.729986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:12.731571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:12.986106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1588df-1518-48f0-b38f-be6da1a8f33b service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:13.000392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:13.001436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:13.001903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:13.002384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] PATH_INFO: `/v3/projects/daf5b93920984d38baa238022c0d464d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:13.006744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf5b93920984d38baa238022c0d464d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:13.009738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:13.026621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:13.194727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf5b93920984d38baa238022c0d464d', 'request_id': 'req-b006667f-b6b3-4923-948e-45c683c7e202'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:13.216913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf5b93920984d38baa238022c0d464d', 'request_id': 'req-b006667f-b6b3-4923-948e-45c683c7e202'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:13.251762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf5b93920984d38baa238022c0d464d', 'request_id': 'req-b006667f-b6b3-4923-948e-45c683c7e202'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:13.271742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] The token cache is being invalidate because project daf5b93920984d38baa238022c0d464d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:13.272275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:13.273425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b006667f-b6b3-4923-948e-45c683c7e202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:13.278869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 864/1747] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:12 2022] DELETE /identity/v3/projects/daf5b93920984d38baa238022c0d464d => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:13.280336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:13.310809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8cc47-f80e-4574-be45-893562c754eb service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:13.706387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:13.712196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:13.712559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:13.712851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] PATH_INFO: `/v3/users/50991aafa169469da59b87b69d96a950` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:13.719033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50991aafa169469da59b87b69d96a950)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:13.725011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:13.815529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:13.816135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:13.816135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:13.970877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:13.979461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50991aafa169469da59b87b69d96a950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:14.005503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50991aafa169469da59b87b69d96a950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:14.007143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50991aafa169469da59b87b69d96a950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:14.008812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50991aafa169469da59b87b69d96a950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:14.042248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50991aafa169469da59b87b69d96a950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:14.051369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:14.053156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:14.112063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 884/1748] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:13 2022] DELETE /identity/v3/users/50991aafa169469da59b87b69d96a950 => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:14.261321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 865/1749] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:14.573104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:14.574582 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:14.861577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89b7a0-145a-49de-92ee-85175286dd76 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:14.885775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedae74-9045-49b6-8bd1-e4bac45b08bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:14.887705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedae74-9045-49b6-8bd1-e4bac45b08bb service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:14.888957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedae74-9045-49b6-8bd1-e4bac45b08bb service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:14.890094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedae74-9045-49b6-8bd1-e4bac45b08bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:15.041629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedae74-9045-49b6-8bd1-e4bac45b08bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:15.044280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedae74-9045-49b6-8bd1-e4bac45b08bb service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:15.067766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 885/1750] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:00:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:00:15.169267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:15.170866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:15.202848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1e814-4c3b-4ba4-afdb-60adf3f0c389 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:15.365873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:15.366556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:15.366837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:15.367117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] PATH_INFO: `/v3/projects/f84bbbe3ee9b4542b76b2580098ab655` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:15.368509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84bbbe3ee9b4542b76b2580098ab655)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:15.370818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:15.391470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:15.510017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84bbbe3ee9b4542b76b2580098ab655', 'request_id': 'req-3e7570cb-5ec5-4944-b466-15f169296988'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:15.539872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84bbbe3ee9b4542b76b2580098ab655', 'request_id': 'req-3e7570cb-5ec5-4944-b466-15f169296988'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:15.570201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84bbbe3ee9b4542b76b2580098ab655', 'request_id': 'req-3e7570cb-5ec5-4944-b466-15f169296988'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:15.597373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] The token cache is being invalidate because project f84bbbe3ee9b4542b76b2580098ab655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:15.598201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:15.599806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3e7570cb-5ec5-4944-b466-15f169296988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:15.607855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 866/1751] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:15 2022] DELETE /identity/v3/projects/f84bbbe3ee9b4542b76b2580098ab655 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:15.609910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:15.610557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:15.610557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:15.669711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:15.674925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:15.809186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:15.870602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 886/1752] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:15.876824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:15.877318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:15.877729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:15.952026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:15.953550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:15.963839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:15.965283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:15.971739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 867/1753] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:15.975600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:15.990144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7c805-de59-4eb8-8ac0-2802afd03002 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:15.994372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cdef5-2e8e-49e0-a917-de8b84a36543 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.002793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.003567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.004020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.004658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.006269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:16.007290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.012585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.037043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.038182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.038182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.038412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] PATH_INFO: `/v3/users/52bd3e0e95d6434e8c120113e5f7f226` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.048024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bd3e0e95d6434e8c120113e5f7f226)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.056836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.074088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 868/1754] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:16.076565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:16.107970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d737e-530a-40cb-a2e0-16b76dbf314a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.123312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f1fd-5aa0-4924-8169-42f3de5e6542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.128228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f1fd-5aa0-4924-8169-42f3de5e6542 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.129211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f1fd-5aa0-4924-8169-42f3de5e6542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.129719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f1fd-5aa0-4924-8169-42f3de5e6542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.135496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f1fd-5aa0-4924-8169-42f3de5e6542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.144968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f1fd-5aa0-4924-8169-42f3de5e6542 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.197366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bd3e0e95d6434e8c120113e5f7f226'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:16.219245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bd3e0e95d6434e8c120113e5f7f226'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:16.220064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bd3e0e95d6434e8c120113e5f7f226'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:16.220878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bd3e0e95d6434e8c120113e5f7f226'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:16.230947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bd3e0e95d6434e8c120113e5f7f226'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:16.238027 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 869/1755] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:16.239429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba532-793d-40ad-8d8d-00e00b5c1c86 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.239723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba532-793d-40ad-8d8d-00e00b5c1c86 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.240006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba532-793d-40ad-8d8d-00e00b5c1c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.270679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 887/1756] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:15 2022] DELETE /identity/v3/users/52bd3e0e95d6434e8c120113e5f7f226 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:16.270679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:16.271966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.286323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4a1ba532-793d-40ad-8d8d-00e00b5c1c86 None None] MFA Rules not processed for user `a22c6bd0fac8443c844d7734aba97ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:16.288849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4a1ba532-793d-40ad-8d8d-00e00b5c1c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.453429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c919f4-7e6f-47c5-ab83-e943cbb45744 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.458837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 870/1757] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:16.460959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.461930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.462478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.474225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.474962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.475250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.475533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.478508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.481670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.521245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:16.523695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.540024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 888/1758] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:16.541749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:16.549444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 871/1759] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:16.572763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034aa1-0079-4b59-b8e9-3e1a32500675 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.589616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.590919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.591343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.591892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.592683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.596277 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.605388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:16.607851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.626824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 889/1760] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:16.631708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:16.662470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c78b-594e-4049-ab39-ee3d75a45831 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.674107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.675103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.675844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.677116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.678089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.680796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.698368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 890/1761] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:16.703503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:16.727588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69422b-b741-478d-b536-761267bfed95 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.747943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87be9b-75d3-4bf5-aa87-9bec96a5cc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.749871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87be9b-75d3-4bf5-aa87-9bec96a5cc75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.750303 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87be9b-75d3-4bf5-aa87-9bec96a5cc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.750875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87be9b-75d3-4bf5-aa87-9bec96a5cc75 admin admin] PATH_INFO: `/v3/projects/e1a88cff42f84c319024f8a837089742/users/02a9558688e2434c9525edfe1bed127d/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.771853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48422a0f-c385-4f14-945c-27efc28ccc4b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:16.772518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87be9b-75d3-4bf5-aa87-9bec96a5cc75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a88cff42f84c319024f8a837089742, user_id=02a9558688e2434c9525edfe1bed127d, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.777181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87be9b-75d3-4bf5-aa87-9bec96a5cc75 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.778448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c501-bb32-4116-ad8c-82d5486191ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:16.780321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c501-bb32-4116-ad8c-82d5486191ac service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.781206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c501-bb32-4116-ad8c-82d5486191ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.781886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c501-bb32-4116-ad8c-82d5486191ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.791773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:16.806578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 891/1762] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:16 2022] PUT /identity/v3/projects/e1a88cff42f84c319024f8a837089742/users/02a9558688e2434c9525edfe1bed127d/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:16.806578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:16.809187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27c501-bb32-4116-ad8c-82d5486191ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:16.809920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.810392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.810817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.811614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27c501-bb32-4116-ad8c-82d5486191ac service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:16.836926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 872/1763] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:16.840921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.841364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.843251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.857979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] MFA Rules not processed for user `16610ae2c3214396b78fb1caff01ab6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:16.860374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.911217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] MFA Rules not processed for user `02a9558688e2434c9525edfe1bed127d`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:16.916346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:17.038654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 892/1764] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:17.065508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 873/1765] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:17.069375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.215127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c7c56-bbe0-44e9-92e8-f9515707fd4a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.231928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.232761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.233048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.233324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.234159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.237609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.320677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 893/1766] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:17 2022] POST /identity/v3/projects => generated 365 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:17.325398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.339819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8e85b-82d5-465a-8472-8b949b8e2713 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.531094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.531859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.532146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.534392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.535094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.537558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.617928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 874/1767] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:17 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:17.622047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.638453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b203d8-2461-41b3-a12c-bd68fa7bcd0d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.657297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.658034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.658319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.659337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.660100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.662287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.695076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.718299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 894/1768] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:17.723419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ddcbc-acd4-40ce-aac1-51a7679bb4bc admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.730843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.731546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.731830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.732114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] PATH_INFO: `/v3/projects/21bdb3b788654b57bab4337723cc0ac9/users/2639eb03ba5c4e02bbb97047cad65b41/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.793049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21bdb3b788654b57bab4337723cc0ac9, user_id=2639eb03ba5c4e02bbb97047cad65b41, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.811374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.869295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.869898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 875/1769] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:17 2022] PUT /identity/v3/projects/21bdb3b788654b57bab4337723cc0ac9/users/2639eb03ba5c4e02bbb97047cad65b41/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:17.898812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26af9-e02f-4b5e-818b-83f960fb7b6d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:18.118940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0fa97-1b4a-4c25-99bc-b20dc1866877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:18.120938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0fa97-1b4a-4c25-99bc-b20dc1866877 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:18.121347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0fa97-1b4a-4c25-99bc-b20dc1866877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:18.121657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0fa97-1b4a-4c25-99bc-b20dc1866877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:18.122359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0fa97-1b4a-4c25-99bc-b20dc1866877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:18.127646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0fa97-1b4a-4c25-99bc-b20dc1866877 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:18.172229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 895/1770] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:18.176947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:18.210731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8f08-1887-4fa9-9265-bfd6abb0c4e4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:18.223998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb090890-414e-4575-ae0e-2792832e7560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:18.225112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb090890-414e-4575-ae0e-2792832e7560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:18.225112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb090890-414e-4575-ae0e-2792832e7560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:18.225426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb090890-414e-4575-ae0e-2792832e7560 admin admin] PATH_INFO: `/v3/projects/21bdb3b788654b57bab4337723cc0ac9/users/2639eb03ba5c4e02bbb97047cad65b41/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:18.236982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb090890-414e-4575-ae0e-2792832e7560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21bdb3b788654b57bab4337723cc0ac9, user_id=2639eb03ba5c4e02bbb97047cad65b41, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:18.244013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb090890-414e-4575-ae0e-2792832e7560 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:18.299865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:18.300169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:18.301153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:18.303565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 876/1771] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:18 2022] PUT /identity/v3/projects/21bdb3b788654b57bab4337723cc0ac9/users/2639eb03ba5c4e02bbb97047cad65b41/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:18.384801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] MFA Rules not processed for user `2639eb03ba5c4e02bbb97047cad65b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:18.386225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:18.391606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:18.533555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:18.534761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:18.535875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:18.653126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] MFA Rules not processed for user `934be918dc8c4fa59988e5bb22e56eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:18.659066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 896/1772] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:18.668899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:18.669627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:18.671316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:18.804414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:18.889726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 877/1773] 10.222.0.44 () {58 vars in 1015 bytes} [Sat Sep 10 21:00:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:18.950712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76ad02-e672-47ec-b7be-be720ac11894 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:18.962229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:18.963182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:18.963664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:18.964132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:18.995103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:18.996761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:19.041748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 897/1774] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:19.136516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:19.142826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:19.154709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:19.172996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f82b7-8c97-4c55-ba31-95ca99e6b769 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:19.180836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098461a-f6cf-4a3e-8a42-122e13f7a097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:19.181319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098461a-f6cf-4a3e-8a42-122e13f7a097 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.181635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098461a-f6cf-4a3e-8a42-122e13f7a097 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.181914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098461a-f6cf-4a3e-8a42-122e13f7a097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.420665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098461a-f6cf-4a3e-8a42-122e13f7a097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:19.423649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098461a-f6cf-4a3e-8a42-122e13f7a097 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:19.437860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90bfc1-b500-4eea-b1c6-6e688e776c4a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:19.451971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:19.452707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.453123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.453527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] PATH_INFO: `/v3/projects/d02039664fe44d1182f931c7f7a6cd02` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.455272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d02039664fe44d1182f931c7f7a6cd02)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:19.460172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:19.484281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 898/1775] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:19.516994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.517448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.520191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.667157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd02039664fe44d1182f931c7f7a6cd02', 'request_id': 'req-67e17508-7595-4e01-911c-fa4ddba0692d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.679887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd02039664fe44d1182f931c7f7a6cd02', 'request_id': 'req-67e17508-7595-4e01-911c-fa4ddba0692d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.693505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd02039664fe44d1182f931c7f7a6cd02', 'request_id': 'req-67e17508-7595-4e01-911c-fa4ddba0692d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.714832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] The token cache is being invalidate because project d02039664fe44d1182f931c7f7a6cd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:19.715312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.716192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.718438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 878/1776] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:19 2022] DELETE /identity/v3/projects/d02039664fe44d1182f931c7f7a6cd02 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:19.719179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:19.722866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:19.723464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] MFA Rules not processed for user `6e24c7d558f74ce181aad463bf18ee52`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:19.734370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:19.870042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e17508-7595-4e01-911c-fa4ddba0692d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:19.939407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:19.940085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.948183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.948326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.958439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:20.024024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.024024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.033638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:20.071513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 879/1777] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:20.072444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:20.082561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:20.114784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 899/1778] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 607 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:20.193577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:20.375373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01cdb5-e430-45f9-93d3-06cad1cc1a92 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:20.410844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:20.412954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.413246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.413556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] PATH_INFO: `/v3/users/588e9ab78e074d028cb1849883f18d10` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.420140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=588e9ab78e074d028cb1849883f18d10)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.422190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.606906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588e9ab78e074d028cb1849883f18d10'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:20.630575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588e9ab78e074d028cb1849883f18d10'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:20.632278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588e9ab78e074d028cb1849883f18d10'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:20.656262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588e9ab78e074d028cb1849883f18d10'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:20.724802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588e9ab78e074d028cb1849883f18d10'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:20.744973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 880/1779] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:20 2022] DELETE /identity/v3/users/588e9ab78e074d028cb1849883f18d10 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:23.245414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:23.246885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:23.291185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24330090-07f7-467c-ab4c-e9560a6b2a64 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:23.524848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:23.526432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:23.530386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:23.531448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:23.531965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:23.532640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] PATH_INFO: `/v3/users/ec0bc5c09ef4418ea01915e68993f31d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:23.534906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0bc5c09ef4418ea01915e68993f31d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:23.537520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:23.568506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304ec7-92c8-4eea-badd-a162b131bb10 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:23.591126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:23.602649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:23.602939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:23.608204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0bc5c09ef4418ea01915e68993f31d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.612419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] PATH_INFO: `/v3/users/02a9558688e2434c9525edfe1bed127d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:23.613994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a9558688e2434c9525edfe1bed127d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:23.618165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:23.621865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0bc5c09ef4418ea01915e68993f31d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.622863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0bc5c09ef4418ea01915e68993f31d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.623892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0bc5c09ef4418ea01915e68993f31d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.644619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4e230abd-d1b4-48f4-87fe-1ef03af8d459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0bc5c09ef4418ea01915e68993f31d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.692107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 900/1780] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:23 2022] DELETE /identity/v3/users/ec0bc5c09ef4418ea01915e68993f31d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:23.700483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:23.701318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:23.701878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:23.747458 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a9558688e2434c9525edfe1bed127d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.765889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:23.770919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a9558688e2434c9525edfe1bed127d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.772318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a9558688e2434c9525edfe1bed127d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.773626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:23.792680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a9558688e2434c9525edfe1bed127d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.831988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d5bfde01-3aa8-4a5c-a68c-48888aac646b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a9558688e2434c9525edfe1bed127d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:23.853009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 881/1781] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:23 2022] DELETE /identity/v3/users/02a9558688e2434c9525edfe1bed127d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:23.857762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:23.858301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:23.859129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:23.918223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:23.920924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:23.933272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:24.002070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 901/1782] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:24.034497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:24.036348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.044779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 882/1783] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:24.064996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:24.100891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:24.102935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.220584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a3853-7425-4a8e-84d5-2a844be9547c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:24.225752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:24.227674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.228041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.228400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:24.241593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:24.251471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72e483-b53f-41f2-8973-870bc7008a52 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:24.255880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:24.265006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fbb7f-8ae8-4e0e-8a93-8e6ec186f1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:24.265685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fbb7f-8ae8-4e0e-8a93-8e6ec186f1ac service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.266473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fbb7f-8ae8-4e0e-8a93-8e6ec186f1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.266473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fbb7f-8ae8-4e0e-8a93-8e6ec186f1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:24.308478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 883/1784] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:24.315485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fbb7f-8ae8-4e0e-8a93-8e6ec186f1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:24.319818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fbb7f-8ae8-4e0e-8a93-8e6ec186f1ac service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:24.333016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:24.334949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.368806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537374c2-e669-4ae7-b1a9-94b287571692 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:24.376038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 902/1785] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:24.383795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:24.384693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.385095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.386603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] PATH_INFO: `/v3/projects/d4f1a2bc9a814824917f8402c02f490d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:24.387554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f1a2bc9a814824917f8402c02f490d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:24.391626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:24.478349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f1a2bc9a814824917f8402c02f490d', 'request_id': 'req-14bb932c-0dce-41bf-90f2-f78186d87d9b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:24.513772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f1a2bc9a814824917f8402c02f490d', 'request_id': 'req-14bb932c-0dce-41bf-90f2-f78186d87d9b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:24.548154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f1a2bc9a814824917f8402c02f490d', 'request_id': 'req-14bb932c-0dce-41bf-90f2-f78186d87d9b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:24.593409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] The token cache is being invalidate because project d4f1a2bc9a814824917f8402c02f490d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:24.593751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:24.596166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14bb932c-0dce-41bf-90f2-f78186d87d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:24.613179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 884/1786] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:24 2022] DELETE /identity/v3/projects/d4f1a2bc9a814824917f8402c02f490d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:24.615548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.615833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.616145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:24.656620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:24.657915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.983714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 903/1787] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:24.992701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.992963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.993315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.076370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:25.084696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:25.128757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 885/1788] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:25.129954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:25.150993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5b8c0-b278-4583-a24b-e016d26922a7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:25.167786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:25.170678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.170972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.174052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.177335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:25.178549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:25.181682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:25.215230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 904/1789] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:00:25.217340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:25.231948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52732d4-9877-4380-b8fe-8c7b9f762e64 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:25.252510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:25.253290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.253628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.253909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.254678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:25.256777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:25.321297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 886/1790] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:25 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:25.325322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:25.371188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24792f-0126-44ce-8356-84b271b360a0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:25.498830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:25.499863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.500359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.500852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.502124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:25.505181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:25.551534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 905/1791] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:25 2022] POST /identity/v3/users => generated 284 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:25.555663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:25.582238 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8ee49-b354-4e9a-b3f7-c0902e10b74b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:25.596010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a7444-b162-4341-bb63-7590ce0a69fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:25.596761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a7444-b162-4341-bb63-7590ce0a69fd admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.620499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a7444-b162-4341-bb63-7590ce0a69fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.621003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a7444-b162-4341-bb63-7590ce0a69fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.622302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a7444-b162-4341-bb63-7590ce0a69fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:25.627450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a7444-b162-4341-bb63-7590ce0a69fd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:25.632715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:25.669197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:25.678085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 887/1792] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:00:25.688447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600a500-6e8a-401f-8fe4-7e769e46685b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:25.720715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc25808-debf-4d59-86c4-5110d6ccfd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:25.721678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc25808-debf-4d59-86c4-5110d6ccfd53 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.721991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc25808-debf-4d59-86c4-5110d6ccfd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.722273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc25808-debf-4d59-86c4-5110d6ccfd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.722937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc25808-debf-4d59-86c4-5110d6ccfd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:25.724943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc25808-debf-4d59-86c4-5110d6ccfd53 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:25.755067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304af976-2811-4bfb-bf14-7d0c8eedb8d6 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.756241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304af976-2811-4bfb-bf14-7d0c8eedb8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.757939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304af976-2811-4bfb-bf14-7d0c8eedb8d6 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.760957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 888/1793] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:00:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:25.771057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 906/1794] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:00:25.775499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304af976-2811-4bfb-bf14-7d0c8eedb8d6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:25.802295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304af976-2811-4bfb-bf14-7d0c8eedb8d6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:25.808281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91751dfe-a04b-4b99-9629-f397b90be101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:25.808965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91751dfe-a04b-4b99-9629-f397b90be101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.809251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91751dfe-a04b-4b99-9629-f397b90be101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.809570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91751dfe-a04b-4b99-9629-f397b90be101 admin admin] PATH_INFO: `/v3/projects/a5d546814bed4046aaceb6a6ffd6316a/users/c3b09264c5be4868a06d1f0eada7abc4/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.869774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91751dfe-a04b-4b99-9629-f397b90be101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d546814bed4046aaceb6a6ffd6316a, user_id=c3b09264c5be4868a06d1f0eada7abc4, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:25.878381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91751dfe-a04b-4b99-9629-f397b90be101 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:25.881630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.881630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.881630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.924158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 889/1795] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:25 2022] PUT /identity/v3/projects/a5d546814bed4046aaceb6a6ffd6316a/users/c3b09264c5be4868a06d1f0eada7abc4/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:25.935175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:25.935384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:25.935982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:25.983115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:25.984443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:26.038253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] MFA Rules not processed for user `c3b09264c5be4868a06d1f0eada7abc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:26.042511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:26.067002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:26.278739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 907/1796] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:00:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:26.290476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 890/1797] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:26.294795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.448028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb431f65-d5ec-40e4-aa26-a671b24555e0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:26.456967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:26.458931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:26.459467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:26.460415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:26.461328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:26.463754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:26.547483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 908/1798] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:26 2022] POST /identity/v3/projects => generated 369 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:26.551584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.571053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57b15-858e-4a1a-a5bf-d0c9443a7108 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:26.728541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:26.730406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:26.731408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:26.732250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:26.733641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:26.736449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:26.781808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.782525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 891/1799] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:26 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:26.783972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.784679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:26.807611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9b2c1-1ae6-45e3-9b07-046125e1a44e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:26.809919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456e0f6-f1ea-4675-a37f-c73ab1ce0b8b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:26.828511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:26.828511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:26.828511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:26.828511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:26.828942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:26.828942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:26.841307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:26.842613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:26.843641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:26.844501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] PATH_INFO: `/v3/projects/e461091b85754b7db5d8f667b14d8373` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:26.848432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e461091b85754b7db5d8f667b14d8373)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:26.854700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:26.866757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 892/1800] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:26.867734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.891564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92e29-7800-41ed-add2-ca158ecc13c8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:26.904951 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:26.905679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:26.905969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:26.906328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] PATH_INFO: `/v3/projects/94dfdedf9d7047c3be2daae1d12f37f9/users/41ba98e111894869ae07dc818df0147b/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:26.937414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dfdedf9d7047c3be2daae1d12f37f9, user_id=41ba98e111894869ae07dc818df0147b, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:26.940674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:26.969715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 893/1801] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:26 2022] PUT /identity/v3/projects/94dfdedf9d7047c3be2daae1d12f37f9/users/41ba98e111894869ae07dc818df0147b/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:26.978714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.996505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e461091b85754b7db5d8f667b14d8373', 'request_id': 'req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.012432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e461091b85754b7db5d8f667b14d8373', 'request_id': 'req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.018291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:27.030153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e461091b85754b7db5d8f667b14d8373', 'request_id': 'req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.078051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] The token cache is being invalidate because project e461091b85754b7db5d8f667b14d8373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:27.084869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.087255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.097994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 909/1802] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:26 2022] DELETE /identity/v3/projects/e461091b85754b7db5d8f667b14d8373 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:27.099242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:27.104778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:27.118301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754042b-fedb-47cc-849c-92f2f5969f21 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:27.149888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:27.221670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:27.222310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:27.222597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:27.222858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:27.223515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:27.225400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:27.263121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 894/1803] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:27.265677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:27.270007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:27.328796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487f36-bb40-420c-a359-dd26c904610f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:27.340180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1288284-18ae-4b4e-b1d6-8d2e918a4e19 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:27.353896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddefb1-d773-414e-bfe4-0ff7b06614a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:27.357833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddefb1-d773-414e-bfe4-0ff7b06614a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:27.359575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddefb1-d773-414e-bfe4-0ff7b06614a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:27.362069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddefb1-d773-414e-bfe4-0ff7b06614a0 admin admin] PATH_INFO: `/v3/projects/94dfdedf9d7047c3be2daae1d12f37f9/users/41ba98e111894869ae07dc818df0147b/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:27.363335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:27.365145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:27.366726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:27.367005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] PATH_INFO: `/v3/users/2ea37598f67e4842ad08ae819e0af322` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:27.370567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddefb1-d773-414e-bfe4-0ff7b06614a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dfdedf9d7047c3be2daae1d12f37f9, user_id=41ba98e111894869ae07dc818df0147b, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:27.374094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea37598f67e4842ad08ae819e0af322)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:27.379507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddefb1-d773-414e-bfe4-0ff7b06614a0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:27.382356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:27.437609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 895/1804] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:27 2022] PUT /identity/v3/projects/94dfdedf9d7047c3be2daae1d12f37f9/users/41ba98e111894869ae07dc818df0147b/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:27.457261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:27.458041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:27.458528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:27.556624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] MFA Rules not processed for user `41ba98e111894869ae07dc818df0147b`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:27.558315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:27.568925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea37598f67e4842ad08ae819e0af322'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.593798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea37598f67e4842ad08ae819e0af322'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.594606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea37598f67e4842ad08ae819e0af322'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.595232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea37598f67e4842ad08ae819e0af322'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.617159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea37598f67e4842ad08ae819e0af322'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:27.699154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 910/1805] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:27 2022] DELETE /identity/v3/users/2ea37598f67e4842ad08ae819e0af322 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:27.700397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:27.704815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:27.765641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 896/1806] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:27.771892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:27.778148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:27.921584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:27.931446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8678c2-abc9-4b53-adfe-238050358ae5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:27.946439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:27.946883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:27.950959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:27.951251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] PATH_INFO: `/v3/projects/e1a88cff42f84c319024f8a837089742` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:27.956877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a88cff42f84c319024f8a837089742)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:27.958896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:27.995092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f671f-5c14-4b66-8ba2-421cf5f3efd1 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:28.130778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:28.133233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:28.133782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:28.135866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:28.139459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:28.170258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a88cff42f84c319024f8a837089742', 'request_id': 'req-5dd97b47-7d03-47d7-815c-b2596613d227'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.183063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a88cff42f84c319024f8a837089742', 'request_id': 'req-5dd97b47-7d03-47d7-815c-b2596613d227'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.196324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a88cff42f84c319024f8a837089742', 'request_id': 'req-5dd97b47-7d03-47d7-815c-b2596613d227'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.215041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] The token cache is being invalidate because project e1a88cff42f84c319024f8a837089742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:28.215436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.217630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.224500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 911/1807] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:27 2022] DELETE /identity/v3/projects/e1a88cff42f84c319024f8a837089742 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:28.228234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:28.230229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:28.340278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:28.342608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:28.379887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 897/1808] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:28.442113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd97b47-7d03-47d7-815c-b2596613d227 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:28.465907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:28.468767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:28.469257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:28.470931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] PATH_INFO: `/v3/users/2639eb03ba5c4e02bbb97047cad65b41` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:28.473943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2639eb03ba5c4e02bbb97047cad65b41)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:28.478293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:28.495114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:28.496990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:28.530745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f5484-c2f4-4846-a87b-efb61e56534e service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:28.590175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2639eb03ba5c4e02bbb97047cad65b41'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.604459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2639eb03ba5c4e02bbb97047cad65b41'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.605530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2639eb03ba5c4e02bbb97047cad65b41'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.606433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2639eb03ba5c4e02bbb97047cad65b41'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.631992 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2639eb03ba5c4e02bbb97047cad65b41'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:28.683902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 912/1809] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:28 2022] DELETE /identity/v3/users/2639eb03ba5c4e02bbb97047cad65b41 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:28.726116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:28.728147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:28.728515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:28.728865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:28.738835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:28.818134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:28.821010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:28.869303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:28.905039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:28.907445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c5315-cf31-450c-b8c3-7d898084451c service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:28.948662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:28.954135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 898/1810] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:29.041583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b832c1-3e61-417f-8e53-933d64c89d11 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:29.075723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:29.076875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:29.077385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:29.077976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:29.085141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:29.117809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:29.121312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:29.158694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 913/1811] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:29.963516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2700eb-ff5e-4d07-81c1-e938dd17e5d7 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:29.964083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2700eb-ff5e-4d07-81c1-e938dd17e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:29.964083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2700eb-ff5e-4d07-81c1-e938dd17e5d7 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:29.966068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 899/1812] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:00:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:29.973697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:29.975798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:30.026627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.027072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.027482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.133295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:30.138326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:30.171482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 900/1813] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:30.222378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898d5a6-1c7e-4da5-a899-e9f0bfa6e591 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:30.237943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:30.238699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.239051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.239399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] PATH_INFO: `/v3/projects/e977c936189a47d0b77fd4e9a0eef203` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.247275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e977c936189a47d0b77fd4e9a0eef203)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:30.250134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:30.307111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:30.308945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:30.344892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2208c-71c5-4445-bdda-2c78ce765130 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:30.353184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fc839-060c-44e6-9be1-837da309af0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:30.354113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc839-060c-44e6-9be1-837da309af0d service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.354460 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc839-060c-44e6-9be1-837da309af0d service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.354809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc839-060c-44e6-9be1-837da309af0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.379179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fc839-060c-44e6-9be1-837da309af0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:30.381432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fc839-060c-44e6-9be1-837da309af0d service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:30.393871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e977c936189a47d0b77fd4e9a0eef203', 'request_id': 'req-c2da2614-af8f-4d6a-ab27-d643c078350c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.406283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e977c936189a47d0b77fd4e9a0eef203', 'request_id': 'req-c2da2614-af8f-4d6a-ab27-d643c078350c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.412514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 901/1814] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:30.434980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e977c936189a47d0b77fd4e9a0eef203', 'request_id': 'req-c2da2614-af8f-4d6a-ab27-d643c078350c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.471066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] The token cache is being invalidate because project e977c936189a47d0b77fd4e9a0eef203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:30.472168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.473929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2da2614-af8f-4d6a-ab27-d643c078350c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.481668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 914/1815] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:29 2022] DELETE /identity/v3/projects/e977c936189a47d0b77fd4e9a0eef203 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:30.484992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.485391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.486064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.523452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:30.524998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:30.682401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 902/1816] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:30.690580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.690678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.691492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.694927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:30.696735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:30.743377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:30.744320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0ab9-812d-497b-a54e-cf8cabe63442 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:30.748853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:30.774591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:30.775297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.776178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.776178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] PATH_INFO: `/v3/projects/21bdb3b788654b57bab4337723cc0ac9` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.782191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 915/1817] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:30.782614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21bdb3b788654b57bab4337723cc0ac9)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:30.784589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:30.786502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:30.830084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1396bf-de00-4aaa-8f8f-a0da8aba2b27 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:30.862394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:30.863024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:30.863405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:30.863567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:30.864212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:30.864720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:30.866388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:30.917269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21bdb3b788654b57bab4337723cc0ac9', 'request_id': 'req-b68c068b-90e4-4bbb-bc6a-8c0c82273800'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.919252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 916/1818] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:00:30.920200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:30.933365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21bdb3b788654b57bab4337723cc0ac9', 'request_id': 'req-b68c068b-90e4-4bbb-bc6a-8c0c82273800'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.948284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0ed10-7d63-47b7-8d3f-9352aad12a50 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:30.951072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21bdb3b788654b57bab4337723cc0ac9', 'request_id': 'req-b68c068b-90e4-4bbb-bc6a-8c0c82273800'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.978639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] The token cache is being invalidate because project 21bdb3b788654b57bab4337723cc0ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:30.979608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.981966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b68c068b-90e4-4bbb-bc6a-8c0c82273800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:30.991772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 903/1819] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:30 2022] DELETE /identity/v3/projects/21bdb3b788654b57bab4337723cc0ac9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:31.004739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee5ca-5210-4f0c-9f1f-dc501d7312f0 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.005032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee5ca-5210-4f0c-9f1f-dc501d7312f0 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.005311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee5ca-5210-4f0c-9f1f-dc501d7312f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.045725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:31.057675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-945ee5ca-5210-4f0c-9f1f-dc501d7312f0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:31.059259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-945ee5ca-5210-4f0c-9f1f-dc501d7312f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:31.107417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.107987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.108267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.108541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.109185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.122528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.126147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:31.152333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 904/1820] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:31.161343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.161723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.162012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.189412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 917/1821] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:30 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:31.194646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.195162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:31.196417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:31.197412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:31.331733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4758092-4e72-4339-b60a-091e736cf140 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.354144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.355136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.355990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.358553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.362045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.366635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.414670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 905/1822] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:31.416505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.441232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0c61-9912-443f-8c96-47586baf95e2 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.454362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 918/1823] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:31.458967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.482476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.483121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.483419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.483688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.491108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:31.491357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.497237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f80a0d-dbbc-4198-b0c2-c0325b9cce7a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.499284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.512049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.512683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.513104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.513486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.514177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.515884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.539759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 919/1824] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:31.544627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 906/1825] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:31.545477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.550316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.573386 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d06149-b87f-4830-961d-f7eaef93608c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.583124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc14289-1758-459d-839d-a139046164f4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.592041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.592288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.592407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.592407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.592407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.592672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.592932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.593044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.594233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.594794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.596810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.599457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.618258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 920/1826] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:00:31.622402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.642103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70413165-870e-4b1e-9c76-0785008160bb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.665646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cddcc-94d0-4657-9054-0a4ce8205a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.666627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cddcc-94d0-4657-9054-0a4ce8205a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.667079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cddcc-94d0-4657-9054-0a4ce8205a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.667355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cddcc-94d0-4657-9054-0a4ce8205a4c admin admin] PATH_INFO: `/v3/projects/cdc2930741774882b13848c4e086d866/users/fb80f30155b64abda72b85a8c8edd9d1/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.704229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cddcc-94d0-4657-9054-0a4ce8205a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc2930741774882b13848c4e086d866, user_id=fb80f30155b64abda72b85a8c8edd9d1, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.705397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 907/1827] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:31.707066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cddcc-94d0-4657-9054-0a4ce8205a4c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.707594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.749545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 921/1828] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:31 2022] PUT /identity/v3/projects/cdc2930741774882b13848c4e086d866/users/fb80f30155b64abda72b85a8c8edd9d1/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:31.753986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.754460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.754877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.811132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] MFA Rules not processed for user `fb80f30155b64abda72b85a8c8edd9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:31.812628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:31.863326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4fce2-1ff2-4028-8c70-a44a32c12501 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.933008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:31.942141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 922/1829] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:31.946671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.947239 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:31.948237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.948526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.948805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.949632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.951658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.959960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecdcc5-95e5-4d39-a53c-423eba95ae63 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:31.966722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:31.967557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.968294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.968804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.969807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:31.971834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:31.972168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:31.985747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 908/1830] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:31.986941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.001017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262b359-64ab-4d1e-918e-2ae94fd5503e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.007020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.009356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.009806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.010293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.011079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.013064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.014372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 923/1831] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:32.018547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.032244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 909/1832] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:32.033584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.050890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20b648-3c7d-40fc-8780-178e7980c63e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.115697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c5df3-babf-44d9-a91b-c89cc9b58c41 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.123310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.124585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.125130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.125747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.125870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.126735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.127053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.127339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.127615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.128261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.129843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.130759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.142863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 910/1833] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:32.147140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.161618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177445-e4c3-4901-9347-812024678ad0 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.165838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 924/1834] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:32 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:32.167511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1718b1-8d49-49b0-bc81-4bdfd9ba1d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.168850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1718b1-8d49-49b0-bc81-4bdfd9ba1d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.169260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1718b1-8d49-49b0-bc81-4bdfd9ba1d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.169641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1718b1-8d49-49b0-bc81-4bdfd9ba1d80 admin admin] PATH_INFO: `/v3/projects/cabc1f2ee4624bd6b506e591376d1aea/users/4facd3f30637423094cbfd870fc4d971/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.170038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.185032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cd33d-fb33-4230-bca7-755dfe593d9f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.191184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1718b1-8d49-49b0-bc81-4bdfd9ba1d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabc1f2ee4624bd6b506e591376d1aea, user_id=4facd3f30637423094cbfd870fc4d971, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.191899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.192867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.193230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.193611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.194506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1718b1-8d49-49b0-bc81-4bdfd9ba1d80 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.194632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.196417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.213208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 925/1835] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:32.217712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.221549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 911/1836] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:32 2022] PUT /identity/v3/projects/cabc1f2ee4624bd6b506e591376d1aea/users/4facd3f30637423094cbfd870fc4d971/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:32.237539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.241062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.241381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.267002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ad5e2-0546-4fb1-a16b-bb46db6184f4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.313599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] MFA Rules not processed for user `4facd3f30637423094cbfd870fc4d971`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:32.315153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:32.375539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.376247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.376510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.376794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] PATH_INFO: `/v3/projects/31e31f60c296452e9b21dfe60d20780d/users/bd846f8b92244be4a07e41658e8cca37/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.400901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e31f60c296452e9b21dfe60d20780d, user_id=bd846f8b92244be4a07e41658e8cca37, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.404921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.419838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 912/1837] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:32.424337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.426132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 926/1838] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:32 2022] PUT /identity/v3/projects/31e31f60c296452e9b21dfe60d20780d/users/bd846f8b92244be4a07e41658e8cca37/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:32.431414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.443676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f37e4-3fd6-45a7-9434-a46706fd59d4 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.504522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:32.524868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:32.541864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6948c7b-10ed-4aa9-aeb3-58c6d0da3d89 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.552730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.553811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.554283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.555165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.556149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.558752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.562250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.562967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.563258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.563535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.564199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.577516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.578342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 927/1839] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:32.585078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.615735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 913/1840] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:32 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:32.619311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b27e7d-37ce-4dce-bf10-ba0594386c5c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.620236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:32.747993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4db4a-27be-43ea-a072-736473875e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.751082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4db4a-27be-43ea-a072-736473875e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.751732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4db4a-27be-43ea-a072-736473875e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.753230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4db4a-27be-43ea-a072-736473875e0e admin admin] PATH_INFO: `/v3/projects/31e31f60c296452e9b21dfe60d20780d/users/bd846f8b92244be4a07e41658e8cca37/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.763603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4db4a-27be-43ea-a072-736473875e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e31f60c296452e9b21dfe60d20780d, user_id=bd846f8b92244be4a07e41658e8cca37, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.775279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4db4a-27be-43ea-a072-736473875e0e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.794987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a067ed8-576b-4e62-9e5e-8421d31437d3 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:32.798990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 928/1841] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:32 2022] PUT /identity/v3/projects/31e31f60c296452e9b21dfe60d20780d/users/bd846f8b92244be4a07e41658e8cca37/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:32.816653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.816653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.816872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.903909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] MFA Rules not processed for user `bd846f8b92244be4a07e41658e8cca37`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:32.905346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:32.963809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:32.964838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:32.966349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:32.967102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:32.967867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:32.970933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:32.999210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 914/1842] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:32 2022] POST /identity/v3/users => generated 298 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:33.002744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.014233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0114-9b56-4dcc-b2f9-378d1ffe9c6c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:33.022411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:33.023468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.023760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.024431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.026498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:33.028140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:33.063667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 915/1843] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:33.065153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.082751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85110d5-edd5-4c77-b016-5a9c660eb8fd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:33.088972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 929/1844] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:33.097855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:33.098551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.098858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.099113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] PATH_INFO: `/v3/projects/70cc77d62b5e45f18d240dca96868e88/users/c198ef7d3b5340a0ad85b1ed445468c5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.110715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.112330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:33.140073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cc77d62b5e45f18d240dca96868e88, user_id=c198ef7d3b5340a0ad85b1ed445468c5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:33.142860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:33.168417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 916/1845] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:33 2022] PUT /identity/v3/projects/70cc77d62b5e45f18d240dca96868e88/users/c198ef7d3b5340a0ad85b1ed445468c5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:33.172792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.217613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f4b23-42ad-4cd5-b117-36c4223c30e3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:33.229214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:33.252428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:33.294145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61574469-7d5e-449b-bca0-6dfce49c9767 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:33.314765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:33.315462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.315846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.316130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.319972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:33.326586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:33.326944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:33.327583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.327863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.328136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.361403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 917/1846] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:00:33.370173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.388442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bac31-4395-4874-bcc5-2db572d29a13 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:33.395427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba7454-f6af-48e1-9a5a-56909b893558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:33.396235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba7454-f6af-48e1-9a5a-56909b893558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.396596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba7454-f6af-48e1-9a5a-56909b893558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.397259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba7454-f6af-48e1-9a5a-56909b893558 admin admin] PATH_INFO: `/v3/projects/70cc77d62b5e45f18d240dca96868e88/users/c198ef7d3b5340a0ad85b1ed445468c5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.400607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba7454-f6af-48e1-9a5a-56909b893558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cc77d62b5e45f18d240dca96868e88, user_id=c198ef7d3b5340a0ad85b1ed445468c5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:33.403271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba7454-f6af-48e1-9a5a-56909b893558 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:33.427046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 918/1847] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:33 2022] PUT /identity/v3/projects/70cc77d62b5e45f18d240dca96868e88/users/c198ef7d3b5340a0ad85b1ed445468c5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:33.438472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.438771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.439242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.439866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:33.441966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:33.472673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 930/1848] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:33.487567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] MFA Rules not processed for user `c198ef7d3b5340a0ad85b1ed445468c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:33.489128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:33.660148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 919/1849] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:33.680906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.682405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:33.713248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f373c-bc62-435c-9b1a-7f7b12eb5485 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:33.818911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:33.819067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:33.819327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:33.819608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:33.832330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:33.840889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:33.872061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 931/1850] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:33.902983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.914395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:33.916359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.096850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7502c-156e-4707-8bd7-063ffe7dc8f2 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.114764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.115435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.115723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.116003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] PATH_INFO: `/v3/users/4facd3f30637423094cbfd870fc4d971` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.120704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4facd3f30637423094cbfd870fc4d971)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.126255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.144403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53574590-92b6-4205-9f75-e63058224ac1 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.160864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.162361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.162705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.163627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] PATH_INFO: `/v3/users/c3b09264c5be4868a06d1f0eada7abc4` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.169672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b09264c5be4868a06d1f0eada7abc4)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.172040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.176624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:34.264535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b09264c5be4868a06d1f0eada7abc4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.269940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:34.273689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4facd3f30637423094cbfd870fc4d971'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.279814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b09264c5be4868a06d1f0eada7abc4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.281078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b09264c5be4868a06d1f0eada7abc4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.282440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b09264c5be4868a06d1f0eada7abc4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.288023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4facd3f30637423094cbfd870fc4d971'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.290920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4facd3f30637423094cbfd870fc4d971'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.296530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b09264c5be4868a06d1f0eada7abc4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.304302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4facd3f30637423094cbfd870fc4d971'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.323171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-676a731e-6a39-41ef-be78-6d21d025a431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4facd3f30637423094cbfd870fc4d971'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:34.334848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 932/1851] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:33 2022] DELETE /identity/v3/users/c3b09264c5be4868a06d1f0eada7abc4 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:34.334848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.337036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.342083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 920/1852] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:33 2022] DELETE /identity/v3/users/4facd3f30637423094cbfd870fc4d971 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:34.344066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b50924-3411-4eaa-ad2d-77b1505519fa None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.345771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b50924-3411-4eaa-ad2d-77b1505519fa None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.346218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b50924-3411-4eaa-ad2d-77b1505519fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.378304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f1b50924-3411-4eaa-ad2d-77b1505519fa None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:34.380892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f1b50924-3411-4eaa-ad2d-77b1505519fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.498041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 921/1853] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:34.498041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.498041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.498041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.506646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37061af2-069e-440e-a916-ef1c11859b5c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.549950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:34.549950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.559904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 922/1854] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:34.559904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.559904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.596365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd846f8b92244be4a07e41658e8cca37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31e31f60c296452e9b21dfe60d20780d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.598493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.598785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.599480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.600835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:34.601429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.603683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.616076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 933/1855] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:34.616798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.619540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.634327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3029531-e167-4ce4-831b-84f0cf585b1f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.638761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.639428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.639751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.640103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.650989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.653744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.655906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:34.665963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 923/1856] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:34.666829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.668610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.684844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27764cf-fe5c-46d2-8c6d-749ae884db99 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.689713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd846f8b92244be4a07e41658e8cca37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31e31f60c296452e9b21dfe60d20780d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.690340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.690630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.690913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.691550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.693733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.698478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b98d6-2c2a-4950-8331-3cf477c6e9b6 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.702487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.703104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.703384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.703658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.715495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.716933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.729281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 934/1857] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:34.755540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 924/1858] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:34 2022] POST /identity/v3/projects => generated 355 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:34.760078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.774459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a65f-8f46-4cc3-b1c3-b88439eadc85 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.845564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd846f8b92244be4a07e41658e8cca37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31e31f60c296452e9b21dfe60d20780d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.846171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.846463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.846743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.847460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:34.847780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.849668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.867509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 935/1859] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:34.916494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.929216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea377d7-385e-4940-bae0-9ac2f425c4dc tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.936111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd846f8b92244be4a07e41658e8cca37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31e31f60c296452e9b21dfe60d20780d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.936785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.937794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.938053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] PATH_INFO: `/v3/projects/0c87b2b48d344892af13c81d9fd66aec` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.939292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0c87b2b48d344892af13c81d9fd66aec)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.941244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.995751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:35.015703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c87b2b48d344892af13c81d9fd66aec', 'request_id': 'req-abf39e13-e151-497b-ba4b-e1ab1541c500'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.025732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c87b2b48d344892af13c81d9fd66aec', 'request_id': 'req-abf39e13-e151-497b-ba4b-e1ab1541c500'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.041985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c87b2b48d344892af13c81d9fd66aec', 'request_id': 'req-abf39e13-e151-497b-ba4b-e1ab1541c500'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.064172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] The token cache is being invalidate because project 0c87b2b48d344892af13c81d9fd66aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:35.064889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.065848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.072546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 925/1860] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:34 2022] DELETE /identity/v3/projects/0c87b2b48d344892af13c81d9fd66aec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:35.316445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:35.318378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:35.461938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79d88b-b077-4069-8d7a-9dbe1908c6c8 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:35.643489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:35.646157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:35.646573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:35.646993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] PATH_INFO: `/v3/users/fb80f30155b64abda72b85a8c8edd9d1` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:35.650548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb80f30155b64abda72b85a8c8edd9d1)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:35.655552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:35.667218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:35.740612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb80f30155b64abda72b85a8c8edd9d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.743213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:35.744751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:35.760226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb80f30155b64abda72b85a8c8edd9d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.761167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb80f30155b64abda72b85a8c8edd9d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.762174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb80f30155b64abda72b85a8c8edd9d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.762976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf39e13-e151-497b-ba4b-e1ab1541c500 tempest-VolumeQuotaClassesTest-959327008 tempest-VolumeQuotaClassesTest-959327008-project-admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:35.789683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4858a508-17ed-44b9-b031-6ccc9b24e525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb80f30155b64abda72b85a8c8edd9d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.791111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:35.796483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:35.797284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:35.797841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] PATH_INFO: `/v3/projects/cabc1f2ee4624bd6b506e591376d1aea` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:35.800920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabc1f2ee4624bd6b506e591376d1aea)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:35.807103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:35.848576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 936/1861] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:35 2022] DELETE /identity/v3/users/fb80f30155b64abda72b85a8c8edd9d1 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:35.850360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:35.850723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:35.851004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:35.893644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:35.895515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:35.911226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabc1f2ee4624bd6b506e591376d1aea', 'request_id': 'req-85652b50-394d-4da7-b21b-263fa10f9c95'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.920692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabc1f2ee4624bd6b506e591376d1aea', 'request_id': 'req-85652b50-394d-4da7-b21b-263fa10f9c95'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.937961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabc1f2ee4624bd6b506e591376d1aea', 'request_id': 'req-85652b50-394d-4da7-b21b-263fa10f9c95'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.954047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] The token cache is being invalidate because project cabc1f2ee4624bd6b506e591376d1aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:35.954344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.955523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:35.958877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 926/1862] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:35 2022] DELETE /identity/v3/projects/cabc1f2ee4624bd6b506e591376d1aea => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:35.965616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:35.969838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:36.002480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 937/1863] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:36.059267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:36.061242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:36.081628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652b50-394d-4da7-b21b-263fa10f9c95 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:36.087797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:36.088989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:36.089377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:36.089860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] PATH_INFO: `/v3/users/c198ef7d3b5340a0ad85b1ed445468c5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:36.091134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c198ef7d3b5340a0ad85b1ed445468c5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:36.094280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:36.159694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca858ff6-c2e2-4290-a5b2-1e8f0eb5b1db None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:36.170999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:36.173401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:36.173610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c198ef7d3b5340a0ad85b1ed445468c5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.174844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:36.175136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:36.175416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:36.191681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c198ef7d3b5340a0ad85b1ed445468c5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.192372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c198ef7d3b5340a0ad85b1ed445468c5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.203764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:36.208282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:36.219069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c198ef7d3b5340a0ad85b1ed445468c5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.225083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 938/1864] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:36.245579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c198ef7d3b5340a0ad85b1ed445468c5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.257119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 927/1865] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:35 2022] DELETE /identity/v3/users/c198ef7d3b5340a0ad85b1ed445468c5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:36.264457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:36.265930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:36.279996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d34c9-d863-4d53-94c9-fb34ebee5bc2 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:36.316752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:36.382114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:36.383980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:36.384975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:36.385477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] PATH_INFO: `/v3/projects/a5d546814bed4046aaceb6a6ffd6316a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:36.387153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d546814bed4046aaceb6a6ffd6316a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:36.394229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:36.529660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d546814bed4046aaceb6a6ffd6316a', 'request_id': 'req-22bcad11-fb09-4133-b481-1f14c1bd556e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.541806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d546814bed4046aaceb6a6ffd6316a', 'request_id': 'req-22bcad11-fb09-4133-b481-1f14c1bd556e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.557767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d546814bed4046aaceb6a6ffd6316a', 'request_id': 'req-22bcad11-fb09-4133-b481-1f14c1bd556e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.574891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] The token cache is being invalidate because project a5d546814bed4046aaceb6a6ffd6316a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:36.575479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.576652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.582588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 939/1866] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:36 2022] DELETE /identity/v3/projects/a5d546814bed4046aaceb6a6ffd6316a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:36.583441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:36.585230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:36.604874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085184-116c-4b22-95d5-bed710d47be4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:36.689398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:36.690026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:36.690573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:36.690896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] PATH_INFO: `/v3/users/41ba98e111894869ae07dc818df0147b` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:36.692210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ba98e111894869ae07dc818df0147b)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:36.694489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:36.758324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ba98e111894869ae07dc818df0147b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.768049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ba98e111894869ae07dc818df0147b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.768777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ba98e111894869ae07dc818df0147b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.790711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ba98e111894869ae07dc818df0147b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.834136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ba98e111894869ae07dc818df0147b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:36.867654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 928/1867] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:36 2022] DELETE /identity/v3/users/41ba98e111894869ae07dc818df0147b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:37.254347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:37.258876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:37.302280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcad11-fb09-4133-b481-1f14c1bd556e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:37.365890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:37.367526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:37.383840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:37.384887 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:37.385235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:37.385613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] PATH_INFO: `/v3/projects/cdc2930741774882b13848c4e086d866` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:37.386827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc2930741774882b13848c4e086d866)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:37.388811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:37.437868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:37.471463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc2930741774882b13848c4e086d866', 'request_id': 'req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.481133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc2930741774882b13848c4e086d866', 'request_id': 'req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.493373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc2930741774882b13848c4e086d866', 'request_id': 'req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.499803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:37.507028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8626356-2bdd-41de-8991-ff51c4b14cd8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:37.519479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] The token cache is being invalidate because project cdc2930741774882b13848c4e086d866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:37.521240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.521240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.531877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 940/1868] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:37 2022] DELETE /identity/v3/projects/cdc2930741774882b13848c4e086d866 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:37.533282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:37.548437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:37.606261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:37.607203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:37.608288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:37.608712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] PATH_INFO: `/v3/projects/70cc77d62b5e45f18d240dca96868e88` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:37.610358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] RBAC: Authorizing `identity:delete_project(project_id=70cc77d62b5e45f18d240dca96868e88)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:37.613186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:37.672242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb4582-f404-4dd7-8458-bca0e4dd6aec admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:37.678165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:37.679483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:37.680356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:37.680706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] PATH_INFO: `/v3/users/bd846f8b92244be4a07e41658e8cca37` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:37.682358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd846f8b92244be4a07e41658e8cca37)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:37.684968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:37.708265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70cc77d62b5e45f18d240dca96868e88', 'request_id': 'req-195fd3cf-78a6-48a1-8078-86a3fa1de82a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.725129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70cc77d62b5e45f18d240dca96868e88', 'request_id': 'req-195fd3cf-78a6-48a1-8078-86a3fa1de82a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.746566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70cc77d62b5e45f18d240dca96868e88', 'request_id': 'req-195fd3cf-78a6-48a1-8078-86a3fa1de82a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.795890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] The token cache is being invalidate because project 70cc77d62b5e45f18d240dca96868e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:37.796939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.797955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.799864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 929/1869] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:37 2022] DELETE /identity/v3/projects/70cc77d62b5e45f18d240dca96868e88 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:37.801449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:37.806275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:37.815952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd846f8b92244be4a07e41658e8cca37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.831480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd846f8b92244be4a07e41658e8cca37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.832379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd846f8b92244be4a07e41658e8cca37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.833294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd846f8b92244be4a07e41658e8cca37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.862785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cd4581f8-d4c4-47d2-8139-61815b0091fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd846f8b92244be4a07e41658e8cca37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:37.886518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 941/1870] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:37 2022] DELETE /identity/v3/users/bd846f8b92244be4a07e41658e8cca37 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:37.888530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c626-0cae-4742-adc1-2df067c48d39 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:37.888920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c626-0cae-4742-adc1-2df067c48d39 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:37.889422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c626-0cae-4742-adc1-2df067c48d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:37.893493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fd3cf-78a6-48a1-8078-86a3fa1de82a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:37.924831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:37.926451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d1f2c626-0cae-4742-adc1-2df067c48d39 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:37.928490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d1f2c626-0cae-4742-adc1-2df067c48d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:37.969970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:37.970682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:37.971002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:37.971512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] PATH_INFO: `/v3/projects/94dfdedf9d7047c3be2daae1d12f37f9` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:37.972577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=94dfdedf9d7047c3be2daae1d12f37f9)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:37.975136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.021521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 942/1871] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:38.029211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb9fc-7029-44cb-8764-8103d6d5f913 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.029535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb9fc-7029-44cb-8764-8103d6d5f913 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.029816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb9fc-7029-44cb-8764-8103d6d5f913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.040528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94dfdedf9d7047c3be2daae1d12f37f9', 'request_id': 'req-c29fb0af-ba49-465c-bfa0-249aac4acc0d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.048974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94dfdedf9d7047c3be2daae1d12f37f9', 'request_id': 'req-c29fb0af-ba49-465c-bfa0-249aac4acc0d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.057688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94dfdedf9d7047c3be2daae1d12f37f9', 'request_id': 'req-c29fb0af-ba49-465c-bfa0-249aac4acc0d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.059799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d38fb9fc-7029-44cb-8764-8103d6d5f913 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:38.061102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d38fb9fc-7029-44cb-8764-8103d6d5f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.069790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:38.074152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] The token cache is being invalidate because project 94dfdedf9d7047c3be2daae1d12f37f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:38.074700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.075978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c29fb0af-ba49-465c-bfa0-249aac4acc0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.085431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 930/1872] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:37 2022] DELETE /identity/v3/projects/94dfdedf9d7047c3be2daae1d12f37f9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:38.091488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e5ac-2b25-4687-9344-98a7ce3b2523 None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.091887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e5ac-2b25-4687-9344-98a7ce3b2523 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.092269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e5ac-2b25-4687-9344-98a7ce3b2523 None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.094709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 931/1873] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:00:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:00:38.096862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.097295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.098214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.127275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:38.128878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.181933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 943/1874] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:38.183524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.184407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.186742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.214894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:38.221126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 932/1875] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:38.221126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.221362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.227682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5277-8602-4142-9c6b-d13be55ab7c2 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.233586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.233896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.234126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.234642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.235536 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:38.235848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.237816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d326d6f-045c-4f1d-ae2d-08b9027bf10a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.253678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 933/1876] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:38.259393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.260038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.260518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.289004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:38.291105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.305890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 934/1877] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:38.306976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.319311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfc0e9-44ba-45ab-aa37-10863c8494fe None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.325840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.326707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.327083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.327462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.328414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.330531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.351703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 944/1878] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:38.353061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.369215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 935/1879] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:38.369215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.372298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.374719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5f5c8-d456-4db1-8528-8cb4ac7e2bf4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.396481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:38.457590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.458236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.458520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.458798 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.459656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:38.459902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.462634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.477636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 945/1880] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:38.481576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.481709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989a9fe-7d0f-464c-946a-806fe3609287 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.496860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.497552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.497832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.498180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.504533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fa5f-624d-43b6-9a08-a3c3218c26e0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.511882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.513231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.513690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.513952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.514397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.515298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.516360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.518898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.527630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 936/1881] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:38.528232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.531639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.559023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d38c39-ca0c-4ca1-b1f4-e6321d780eb1 service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.565222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.566086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.566534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.566960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] PATH_INFO: `/v3/projects/31e31f60c296452e9b21dfe60d20780d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.568033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 946/1882] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/users => generated 283 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:38.569094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.569584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e31f60c296452e9b21dfe60d20780d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.571725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.596585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07448bac-25d0-47c3-81dc-314152411b3a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.608062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.610031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.610797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.611067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.615066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.621573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.653486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e31f60c296452e9b21dfe60d20780d', 'request_id': 'req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.663267 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:38.665978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e31f60c296452e9b21dfe60d20780d', 'request_id': 'req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.681506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e31f60c296452e9b21dfe60d20780d', 'request_id': 'req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.689163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 947/1883] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:38.689163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.704643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] The token cache is being invalidate because project 31e31f60c296452e9b21dfe60d20780d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:38.705360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.706289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:38.708347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 937/1884] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:38 2022] DELETE /identity/v3/projects/31e31f60c296452e9b21dfe60d20780d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:38.709308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.712196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.771535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7120-d09d-44d5-b0b0-43dd301e9814 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.778811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ff71-8fe1-40e2-8087-fc797c6685c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.779501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ff71-8fe1-40e2-8087-fc797c6685c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.779714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ff71-8fe1-40e2-8087-fc797c6685c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.779999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ff71-8fe1-40e2-8087-fc797c6685c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.780647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150ff71-8fe1-40e2-8087-fc797c6685c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.782354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150ff71-8fe1-40e2-8087-fc797c6685c6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.796335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 948/1885] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:38.797735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148c54-c164-411e-87a9-65452200803e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.798026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148c54-c164-411e-87a9-65452200803e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.798308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148c54-c164-411e-87a9-65452200803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.821625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a7148c54-c164-411e-87a9-65452200803e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:38.823722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a7148c54-c164-411e-87a9-65452200803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.830619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331151f-cae6-4fb0-bf70-b1c1e3cdb357 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.840465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 949/1886] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:38.840465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148c54-c164-411e-87a9-65452200803e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.840465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a7148c54-c164-411e-87a9-65452200803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.840811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f3c9c-8955-498b-adb1-13c23a2062d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.840811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f3c9c-8955-498b-adb1-13c23a2062d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.840811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f3c9c-8955-498b-adb1-13c23a2062d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.840811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f3c9c-8955-498b-adb1-13c23a2062d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.841075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f3c9c-8955-498b-adb1-13c23a2062d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.845285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f3c9c-8955-498b-adb1-13c23a2062d4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.847906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148c54-c164-411e-87a9-65452200803e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.852765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.853390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.853736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.864845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.866264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.868260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.877367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 938/1887] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:38.883173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.883546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.884662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.888654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 950/1888] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:38.888654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.909578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2a19e-4de8-4b82-9c73-8c57c2578602 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.914493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.915120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.915403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.915743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.916395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.917090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:38.918052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.918529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:38.928306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 939/1889] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:38.930154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 951/1890] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:38.930268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.933761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.944872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b7291-3a56-47e2-9e1d-b0a51811024d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.945871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516399-d3c9-4a9b-89c7-92e0ed88e20e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.951049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.951566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.951734 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.952021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.952302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] PATH_INFO: `/v3/projects/c033f650995646bcbdad72ba6dac3dbd/users/0f260ed0192d435dbbf609ec58d6c821/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.952892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.953320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.953797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.954589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:38.955043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.955306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:38.957056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.972475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c033f650995646bcbdad72ba6dac3dbd, user_id=0f260ed0192d435dbbf609ec58d6c821, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:38.973111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 952/1891] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:38.974254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:38.975733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:38.988247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c964b-3630-4232-81f5-3e8e6eae603f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:38.996518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f1f2e-2265-4c74-a392-6c28a66a162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:38.997362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f1f2e-2265-4c74-a392-6c28a66a162b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:38.998481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f1f2e-2265-4c74-a392-6c28a66a162b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:38.998771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f1f2e-2265-4c74-a392-6c28a66a162b admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:38.999426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f1f2e-2265-4c74-a392-6c28a66a162b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.009517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 940/1892] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:38 2022] PUT /identity/v3/projects/c033f650995646bcbdad72ba6dac3dbd/users/0f260ed0192d435dbbf609ec58d6c821/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:39.009890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f1f2e-2265-4c74-a392-6c28a66a162b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.009975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.016525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 953/1893] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:39.017712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.018008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.018291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.069562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] MFA Rules not processed for user `0f260ed0192d435dbbf609ec58d6c821`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:39.071957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:39.111141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00938434-fc0a-43a8-8359-63033d4b3bf2 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.129920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.130286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.130590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.130856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.131731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.136010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.162861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 954/1894] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:39.164641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.176622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f38c-7f5d-4267-8d66-f4c95d47969f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.182946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.183961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.184638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.185135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] PATH_INFO: `/v3/projects/e8e0c2743a3c4495a0d9a1edecbf534b/users/9979f803da8040dfa9ba4bfca12a22a5/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.190616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 941/1895] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:39.191901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.211539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e0c2743a3c4495a0d9a1edecbf534b, user_id=9979f803da8040dfa9ba4bfca12a22a5, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.215030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.233284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 955/1896] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:39 2022] PUT /identity/v3/projects/e8e0c2743a3c4495a0d9a1edecbf534b/users/9979f803da8040dfa9ba4bfca12a22a5/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:39.234640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.244802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:39.335270 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be623-0fec-439c-8848-cd49edac8309 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.345824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c62363-e422-440f-abf4-13875d0c861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.346669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62363-e422-440f-abf4-13875d0c861e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.346969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62363-e422-440f-abf4-13875d0c861e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.347258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62363-e422-440f-abf4-13875d0c861e admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.347913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62363-e422-440f-abf4-13875d0c861e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.350851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62363-e422-440f-abf4-13875d0c861e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.352741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344499e6-747e-45ce-863c-d841e10adad5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.369216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.371993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.372486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.372978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.373845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.376231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.396411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 942/1897] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:39.403752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.404493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.406069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.425320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 956/1898] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:39.426431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.446563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9994871-787f-4f4f-8d85-75f9b420639d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.452153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] MFA Rules not processed for user `9979f803da8040dfa9ba4bfca12a22a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:39.454756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:39.547592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.548890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.549359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.549846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.550909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.555687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.557981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:39.611554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 957/1899] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:39.614722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.620292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 943/1900] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:39.621381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.640524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c3d9-4e87-4f72-b3f6-79d10bd9de32 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.672743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7819-3fb4-4e71-a23f-c9a9a2a6f310 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.679640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.680175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.680337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.680613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.681275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.681738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.682541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.683058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.683227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.684173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.685049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.686993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.698960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 944/1901] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:39.699794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.712401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 958/1902] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:39.713448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.720022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3f938-bab8-4475-87b5-24926007bd91 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.725091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722d00b-5e74-42f3-b3a5-866520971df8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.732336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.733188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.733505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.733793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] PATH_INFO: `/v3/projects/91c35c3c3c28403bb3f03a7029c13861/users/718fe034a686499ea4c49c4ba6f67601/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.738069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.738778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.739153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.739500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.740282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.743105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.757060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c35c3c3c28403bb3f03a7029c13861, user_id=718fe034a686499ea4c49c4ba6f67601, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.762132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.784195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 959/1903] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:39 2022] PUT /identity/v3/projects/91c35c3c3c28403bb3f03a7029c13861/users/718fe034a686499ea4c49c4ba6f67601/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:39.785111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.815781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 945/1904] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:39.816683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.885177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:39.910437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd17ef-27ae-4fad-a6cf-ab8bcd798d01 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.914629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c96e9-1a93-4fd6-9917-47fb67f5f7fb admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.918080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.919056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.920108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.920387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.920679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.920878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.921166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.921390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.921596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.922385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.923037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.924479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.934553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 960/1905] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:39.938256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 946/1906] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:39.939105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.941732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:39.953904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38da9f-c16e-4c59-9567-e7f674718302 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.958285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463b9a-5627-49a5-b59e-94a9b8272a12 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:39.970158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.972231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.973047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.973520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:39.974447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:39.978454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:39.983363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:39.984442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:39.984680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:39.985072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] PATH_INFO: `/v3/projects/d6f328438859480991b48361b2bb3b48/users/c6772a2a5981458b89aa96be9192aa14/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.011831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f328438859480991b48361b2bb3b48, user_id=c6772a2a5981458b89aa96be9192aa14, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.015098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.020126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 961/1907] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:39 2022] POST /identity/v3/users => generated 293 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:40.025097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.035897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819903-184b-4853-94ed-6362a1700d2d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.040804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 947/1908] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:39 2022] PUT /identity/v3/projects/d6f328438859480991b48361b2bb3b48/users/c6772a2a5981458b89aa96be9192aa14/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:40.041794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.115906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15689d9-2241-45c9-9e3c-77181882b943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.116596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15689d9-2241-45c9-9e3c-77181882b943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.117065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15689d9-2241-45c9-9e3c-77181882b943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.118048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15689d9-2241-45c9-9e3c-77181882b943 admin admin] PATH_INFO: `/v3/projects/91c35c3c3c28403bb3f03a7029c13861/users/718fe034a686499ea4c49c4ba6f67601/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.120632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15689d9-2241-45c9-9e3c-77181882b943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c35c3c3c28403bb3f03a7029c13861, user_id=718fe034a686499ea4c49c4ba6f67601, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.123775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15689d9-2241-45c9-9e3c-77181882b943 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.155062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 962/1909] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:40 2022] PUT /identity/v3/projects/91c35c3c3c28403bb3f03a7029c13861/users/718fe034a686499ea4c49c4ba6f67601/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:40.155062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.155062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.155062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.171375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de054-3c38-4968-94ed-5992441428e5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.201479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] MFA Rules not processed for user `c6772a2a5981458b89aa96be9192aa14`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:40.202952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:40.207455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:40.252298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f8b1-d24a-420e-bb58-d373b8503b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.253076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f8b1-d24a-420e-bb58-d373b8503b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.253239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f8b1-d24a-420e-bb58-d373b8503b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.253719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f8b1-d24a-420e-bb58-d373b8503b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.254480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f8b1-d24a-420e-bb58-d373b8503b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.256119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f8b1-d24a-420e-bb58-d373b8503b0d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.276621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 948/1910] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:40.276621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.276851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.277110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.299651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 963/1911] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:40.299651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.310896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] MFA Rules not processed for user `718fe034a686499ea4c49c4ba6f67601`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:40.311080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55b4d6-48b5-4418-bb3f-de70a8c063c4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.314630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:40.321758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.322826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.323114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.323455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.324026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.327819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.356170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 964/1912] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:40.359644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.385780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d82f0b-5d7c-400a-8d85-275ce2232ee3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.397125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.397789 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.398842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.399271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.399898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.410430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.454377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 949/1913] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:40.455320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.458148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 965/1914] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:40.462908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.469231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc894b5-4fb7-49fb-93ce-c06a9486a919 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.513881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:40.565896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.566591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.566875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.567151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] PATH_INFO: `/v3/projects/ee82814e1e5141b1b08c095fbff51d3c/users/0d5620eea9074ecaa4b8fa46b9b4d47b/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.588613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee82814e1e5141b1b08c095fbff51d3c, user_id=0d5620eea9074ecaa4b8fa46b9b4d47b, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.591384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.607135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadee32-d573-4514-8d5c-5b48b07c467b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.613703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 950/1915] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:40 2022] PUT /identity/v3/projects/ee82814e1e5141b1b08c095fbff51d3c/users/0d5620eea9074ecaa4b8fa46b9b4d47b/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:40.614521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.621892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:40.674618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:40.696679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82e063-785e-483b-bb52-20cb9851ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.697573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82e063-785e-483b-bb52-20cb9851ec53 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.697941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82e063-785e-483b-bb52-20cb9851ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.698287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82e063-785e-483b-bb52-20cb9851ec53 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.699085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82e063-785e-483b-bb52-20cb9851ec53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.701074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82e063-785e-483b-bb52-20cb9851ec53 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.701452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847675b6-f342-4e1b-867a-a05dd6ba8ff2 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.705494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.706264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.706533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.706822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.723741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 966/1916] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/users => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:40.725101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.725503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.725987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.756367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] MFA Rules not processed for user `0d5620eea9074ecaa4b8fa46b9b4d47b`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:40.758564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:40.780867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.782476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.795619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 951/1917] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:40.796237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.810109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98079-a02f-46c7-b5d7-4ee11da85408 service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.816229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.816853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.817126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.817405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.818610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.822024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.849527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 952/1918] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:40.850638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.857845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 967/1919] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:40.860330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.890351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485823c6-b73a-41bb-8c86-fd6b38dd0635 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.906566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.907206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.907509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.907760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.908406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.910368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.915579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3da08-dcd3-4ea0-ac0d-047f73e6d1da admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:40.925043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:40.925772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:40.926086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:40.926333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] PATH_INFO: `/v3/projects/6806dd1c65ee4f53ad4c7681f05fa828/users/525a0167dd174e2ba749f19794fbc0a9/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:40.960029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6806dd1c65ee4f53ad4c7681f05fa828, user_id=525a0167dd174e2ba749f19794fbc0a9, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:40.964592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:40.984876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 968/1920] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:40.988904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:40.991056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 953/1921] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:40 2022] PUT /identity/v3/projects/6806dd1c65ee4f53ad4c7681f05fa828/users/525a0167dd174e2ba749f19794fbc0a9/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:40.992945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.044182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:41.106107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c7284-2415-4404-ad4e-091abc1cb014 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.106851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be3ad5-626a-41e9-b314-4f5a25e146a0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.112103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.113017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.113301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.113624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.190551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:41.204575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.205229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.205566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.205850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.206903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.209829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.211770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.214170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.233576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 969/1922] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:41.233576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.237657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 954/1923] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:40 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:41.239726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.248870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26022b58-49d8-454d-80d5-3dc9d0c18229 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.251757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945cf7e-fef6-440f-b759-4ae36c7d8e92 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.256457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.258413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.258637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.259060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.259710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.260050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.260336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.260764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.261166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.261430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.263683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.264133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.280777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 955/1924] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:41.281966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.286091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 970/1925] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:41.290151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.294623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543f4a7-b3a5-4278-af68-3d29612afbec admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.302422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.303211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.303498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.303844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] PATH_INFO: `/v3/projects/58367b2c2e9048b1b684794c393997cf/users/8b7caca316ab4a13bf75d6f5b0e538e5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.306753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d90b-a2b5-4153-ab26-63b5ed3a7478 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.315337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd827205-fc34-4a02-a2f4-f2eec13960ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.317575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd827205-fc34-4a02-a2f4-f2eec13960ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.318170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd827205-fc34-4a02-a2f4-f2eec13960ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.318882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd827205-fc34-4a02-a2f4-f2eec13960ad admin admin] PATH_INFO: `/v3/projects/6806dd1c65ee4f53ad4c7681f05fa828/users/525a0167dd174e2ba749f19794fbc0a9/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.322118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd827205-fc34-4a02-a2f4-f2eec13960ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6806dd1c65ee4f53ad4c7681f05fa828, user_id=525a0167dd174e2ba749f19794fbc0a9, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.325402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd827205-fc34-4a02-a2f4-f2eec13960ad admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.326003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58367b2c2e9048b1b684794c393997cf, user_id=8b7caca316ab4a13bf75d6f5b0e538e5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.328866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.340441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 971/1926] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:41 2022] PUT /identity/v3/projects/6806dd1c65ee4f53ad4c7681f05fa828/users/525a0167dd174e2ba749f19794fbc0a9/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:41.353750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 956/1927] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:41 2022] PUT /identity/v3/projects/58367b2c2e9048b1b684794c393997cf/users/8b7caca316ab4a13bf75d6f5b0e538e5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:41.356197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.356512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.356829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.357735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.424209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] MFA Rules not processed for user `525a0167dd174e2ba749f19794fbc0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:41.425352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:41.449407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43eb08-56a6-4bf8-9785-11fae5714768 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.456082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.456803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.457314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.457660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.458505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.460380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.475241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 957/1928] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:41.479702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.495332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be06-8ed4-4f6f-94d2-bafed2e87543 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.500983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6cc3d-1ffb-421d-92ab-fbf230e6c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.501746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6cc3d-1ffb-421d-92ab-fbf230e6c656 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.502293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6cc3d-1ffb-421d-92ab-fbf230e6c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.502502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6cc3d-1ffb-421d-92ab-fbf230e6c656 admin admin] PATH_INFO: `/v3/projects/58367b2c2e9048b1b684794c393997cf/users/8b7caca316ab4a13bf75d6f5b0e538e5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.504668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6cc3d-1ffb-421d-92ab-fbf230e6c656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58367b2c2e9048b1b684794c393997cf, user_id=8b7caca316ab4a13bf75d6f5b0e538e5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.508478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6cc3d-1ffb-421d-92ab-fbf230e6c656 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.529558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 958/1929] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:41 2022] PUT /identity/v3/projects/58367b2c2e9048b1b684794c393997cf/users/8b7caca316ab4a13bf75d6f5b0e538e5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:41.536755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 972/1930] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:41.537864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.538277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.538633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.550588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.552182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:41.594373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] MFA Rules not processed for user `8b7caca316ab4a13bf75d6f5b0e538e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:41.596390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:41.622381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:41.651598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9d14e-dbc4-453f-9133-d91245a7459f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.722786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.723720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.724621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.724621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.762017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 959/1931] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:41.777832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:41.789575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162721b-c10b-4031-9d55-9ed1ea419d1e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:41.796208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:41.797106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:41.797758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:41.798212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:41.821687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.823642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.838012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 960/1932] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:41.870177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:41.873793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:41.911820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 973/1933] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:42.045029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:42.052605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:42.058680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda240a-73f9-4d91-a994-99bb2e6490d8 service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:42.066322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:42.067033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:42.067624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:42.068421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:42.194106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:42.196167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:42.206879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 961/1934] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:42.560727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:42.675670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c352-ae18-401e-b7d9-27b17a3f94b0 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:42.680523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:42.681221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:42.681650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:42.682005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] PATH_INFO: `/v3/users/c6772a2a5981458b89aa96be9192aa14` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:42.683182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6772a2a5981458b89aa96be9192aa14)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:42.685286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:42.696282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:42.705976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e4db9-48c4-4c79-b77f-b5bcdad6b5b5 service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:42.711828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:42.711978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:42.712184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:42.712419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:42.760418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:42.762465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6772a2a5981458b89aa96be9192aa14'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.775577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6772a2a5981458b89aa96be9192aa14'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.776529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6772a2a5981458b89aa96be9192aa14'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.777373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6772a2a5981458b89aa96be9192aa14'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.788212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4b457a2-81b9-4c7f-a66a-fcf52a78a1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6772a2a5981458b89aa96be9192aa14'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.805451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:42.807915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:42.810920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 974/1935] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:42 2022] DELETE /identity/v3/users/c6772a2a5981458b89aa96be9192aa14 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:42.817078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:42.817405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:42.817678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:42.823435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 962/1936] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:42.871867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:42.873763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:42.975997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 975/1937] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:42.990876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:42.993681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:43.169888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72746e43-dafd-4d3d-be9e-acc3c4b5a76e service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:43.348591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:43.349688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:43.352000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:43.352340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:43.381005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:43.385878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:43.398880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:43.411706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 963/1938] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:43.844627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:43.846283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:43.871611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51365-cc30-4a9f-8676-fbefd86fcef8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:43.908988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:43.915379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:43.915672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:43.916237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] PATH_INFO: `/v3/projects/d6f328438859480991b48361b2bb3b48` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:43.917574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f328438859480991b48361b2bb3b48)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:43.919874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:43.995888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f328438859480991b48361b2bb3b48', 'request_id': 'req-de3d04d1-73fc-4913-a7e0-853f8e273ac7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.010099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f328438859480991b48361b2bb3b48', 'request_id': 'req-de3d04d1-73fc-4913-a7e0-853f8e273ac7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.026211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f328438859480991b48361b2bb3b48', 'request_id': 'req-de3d04d1-73fc-4913-a7e0-853f8e273ac7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.044672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] The token cache is being invalidate because project d6f328438859480991b48361b2bb3b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:44.045285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.046300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.050259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 976/1939] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:43 2022] DELETE /identity/v3/projects/d6f328438859480991b48361b2bb3b48 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:44.054081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:44.067720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fb2af-b7e7-49b4-8ea7-4624ce29e7c4 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:44.163034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:44.164852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.165136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.165416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] PATH_INFO: `/v3/users/525a0167dd174e2ba749f19794fbc0a9` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.167680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525a0167dd174e2ba749f19794fbc0a9)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:44.169862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:44.233835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525a0167dd174e2ba749f19794fbc0a9'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.245907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525a0167dd174e2ba749f19794fbc0a9'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.246476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525a0167dd174e2ba749f19794fbc0a9'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.255014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525a0167dd174e2ba749f19794fbc0a9'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.271387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a5ad69b-0594-4714-af71-fe39ac082e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525a0167dd174e2ba749f19794fbc0a9'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.281612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 964/1940] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:44 2022] DELETE /identity/v3/users/525a0167dd174e2ba749f19794fbc0a9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:44.805853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:44.807449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:44.822014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d04d1-73fc-4913-a7e0-853f8e273ac7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:44.936393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:44.957256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:44.958102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.958449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.958796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] PATH_INFO: `/v3/projects/6806dd1c65ee4f53ad4c7681f05fa828` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.960089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6806dd1c65ee4f53ad4c7681f05fa828)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:44.962890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.024384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6806dd1c65ee4f53ad4c7681f05fa828', 'request_id': 'req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:45.032477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6806dd1c65ee4f53ad4c7681f05fa828', 'request_id': 'req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:45.047547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6806dd1c65ee4f53ad4c7681f05fa828', 'request_id': 'req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:45.064262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] The token cache is being invalidate because project 6806dd1c65ee4f53ad4c7681f05fa828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:45.064563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:45.065378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f4fcdebb-7b06-4b41-a733-e6c3438a0b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:45.071282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 977/1941] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:44 2022] DELETE /identity/v3/projects/6806dd1c65ee4f53ad4c7681f05fa828 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:45.072966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.073264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.073599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.102919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:45.104134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:45.177877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 965/1942] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:45.185833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.186205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.186482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.209759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:45.211786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:45.224812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 978/1943] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:45.226170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.237889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5e518-14fb-4386-96ca-740e3b5daf63 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.244362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.245311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.245634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.245904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.246823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:45.247479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.249204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.262148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 966/1944] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:45.270102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.291785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d18d18-6433-4676-8493-3b4fee77e626 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.298802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.299560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.300079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.300263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.300963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.303480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.347911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 979/1945] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:45 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:45.352942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.363876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1e02c-63c5-4f33-a19e-5d39be85e5df admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.373095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:45.454493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.455040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.455808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.455808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.456861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.458316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.483918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 967/1946] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:45 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:45.488487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.509639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ef3d0-f8db-4943-8594-3f28a2c7a41f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.515392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.515917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.516199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.516478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.517131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.518788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.533646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 980/1947] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:45.535632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.548399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f123a-4363-4c28-a20e-a10f0b1cbcea admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.553204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165ae8-6781-4569-95fc-09d9915f375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.553900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165ae8-6781-4569-95fc-09d9915f375c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.554188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165ae8-6781-4569-95fc-09d9915f375c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.555110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165ae8-6781-4569-95fc-09d9915f375c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.555728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165ae8-6781-4569-95fc-09d9915f375c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.558924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165ae8-6781-4569-95fc-09d9915f375c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.588462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 968/1948] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:45.590512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.601875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570c3e5-e8f2-48ae-a557-b6fa3218438e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.607202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.607904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.608278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.608565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] PATH_INFO: `/v3/projects/ff8220c0fada4f9696f946260903492e/users/9829ff4e240343c1b2681820ccf06c7c/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.625718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8220c0fada4f9696f946260903492e, user_id=9829ff4e240343c1b2681820ccf06c7c, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.628491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.645636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 981/1949] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:45 2022] PUT /identity/v3/projects/ff8220c0fada4f9696f946260903492e/users/9829ff4e240343c1b2681820ccf06c7c/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:45.657447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.657846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.658133 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.690922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] MFA Rules not processed for user `9829ff4e240343c1b2681820ccf06c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:45.692186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:45.774364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 969/1950] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:45.791137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.792912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:45.916266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d083-4e9b-45d5-ba34-a05a8963e25a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.927259 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b8179-a7f3-4180-9e06-dfb0a7e3c07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.927827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b8179-a7f3-4180-9e06-dfb0a7e3c07c service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.928101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b8179-a7f3-4180-9e06-dfb0a7e3c07c service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.928381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b8179-a7f3-4180-9e06-dfb0a7e3c07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.950481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b8179-a7f3-4180-9e06-dfb0a7e3c07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.951647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b8179-a7f3-4180-9e06-dfb0a7e3c07c service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.983828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 982/1951] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:46.900665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:46.902286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:46.981258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006235af-da1a-4b8a-822c-3ff193d953f1 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:46.985667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:46.986332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:46.986624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:46.986885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] PATH_INFO: `/v3/users/0f260ed0192d435dbbf609ec58d6c821` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:46.988000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f260ed0192d435dbbf609ec58d6c821)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:46.990241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:47.008991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:47.057109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f260ed0192d435dbbf609ec58d6c821'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.064757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f260ed0192d435dbbf609ec58d6c821'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.065395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f260ed0192d435dbbf609ec58d6c821'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.073962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f260ed0192d435dbbf609ec58d6c821'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.089336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f260ed0192d435dbbf609ec58d6c821'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.096439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 970/1952] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:46 2022] DELETE /identity/v3/users/0f260ed0192d435dbbf609ec58d6c821 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:47.101495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:47.103132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:47.103594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:47.134660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:47.136080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:47.149285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:47.219859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 983/1953] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:47.230896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:47.232459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:47.293514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1f1a3-f3db-4bb7-8f48-4934273e3052 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:47.297371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:47.298139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:47.298427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:47.298697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:47.308673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:47.310163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:47.320794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 971/1954] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:47.635433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:47.637184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:47.669005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e736d2f-0e06-494c-88f9-d6eb364c330f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:47.675617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:47.676273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:47.677210 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:47.677346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] PATH_INFO: `/v3/projects/c033f650995646bcbdad72ba6dac3dbd` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:47.678584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c033f650995646bcbdad72ba6dac3dbd)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:47.681324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:47.770532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c033f650995646bcbdad72ba6dac3dbd', 'request_id': 'req-22cd1a26-16b7-4952-916e-62e04d8e28ee'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.782816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c033f650995646bcbdad72ba6dac3dbd', 'request_id': 'req-22cd1a26-16b7-4952-916e-62e04d8e28ee'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.807637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c033f650995646bcbdad72ba6dac3dbd', 'request_id': 'req-22cd1a26-16b7-4952-916e-62e04d8e28ee'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.827214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] The token cache is being invalidate because project c033f650995646bcbdad72ba6dac3dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:47.827558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.828477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.830399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 984/1955] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:47 2022] DELETE /identity/v3/projects/c033f650995646bcbdad72ba6dac3dbd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:47.834617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:47.849450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7598-d3ea-4455-a7db-0ff873dabc58 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:47.991277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:47.998737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:47.999356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:48.000017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:48.000425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] PATH_INFO: `/v3/users/718fe034a686499ea4c49c4ba6f67601` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:48.000990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:48.002503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] RBAC: Authorizing `identity:delete_user(user_id=718fe034a686499ea4c49c4ba6f67601)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:48.006604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:48.115025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:48.141499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718fe034a686499ea4c49c4ba6f67601'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.155574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd1a26-16b7-4952-916e-62e04d8e28ee admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:48.166372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:48.167105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:48.167502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:48.167846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:48.176823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718fe034a686499ea4c49c4ba6f67601'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.178384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718fe034a686499ea4c49c4ba6f67601'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.180414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:48.205582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718fe034a686499ea4c49c4ba6f67601'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.280495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718fe034a686499ea4c49c4ba6f67601'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.311248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 972/1956] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:47 2022] DELETE /identity/v3/users/718fe034a686499ea4c49c4ba6f67601 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:48.361930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:48.368702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:48.373634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d73f0-f2df-4d6c-abd5-52e98704beef service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:48.400050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 985/1957] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:48.667082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:48.668706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:48.770274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a116e67-57d2-4bcc-b717-523e036e059a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:48.775580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:48.776222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:48.776501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:48.776775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] PATH_INFO: `/v3/projects/91c35c3c3c28403bb3f03a7029c13861` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:48.777961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c35c3c3c28403bb3f03a7029c13861)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:48.779903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:48.836634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c35c3c3c28403bb3f03a7029c13861', 'request_id': 'req-d676cc2d-4506-4893-8939-7d7c49f84481'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.843754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c35c3c3c28403bb3f03a7029c13861', 'request_id': 'req-d676cc2d-4506-4893-8939-7d7c49f84481'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.853155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c35c3c3c28403bb3f03a7029c13861', 'request_id': 'req-d676cc2d-4506-4893-8939-7d7c49f84481'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.864987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] The token cache is being invalidate because project 91c35c3c3c28403bb3f03a7029c13861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:48.865311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.865990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d676cc2d-4506-4893-8939-7d7c49f84481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:48.867942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 973/1958] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:48 2022] DELETE /identity/v3/projects/91c35c3c3c28403bb3f03a7029c13861 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:48.872495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:48.873119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:48.873406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:48.902556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:48.903914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:48.969093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 986/1959] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:48.977768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:48.978113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:48.978398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:48.999134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:49.000369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:49.009644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 974/1960] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:49.014080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.025169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b141a1-d6ea-49eb-ae30-c29c18c00908 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.030020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.030641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.030922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.031205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.031856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:49.032161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.033804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.046372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 987/1961] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:49.050856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.060977 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12931e-a362-4b66-851e-0e6b51ac751d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.065989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.066610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.066895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.067170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.067824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.070292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.103775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 975/1962] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:49.105056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.115885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113377c-86f1-4682-b61f-09a4fc866802 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.181747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.182437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.182657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.182936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.183575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.185553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.209703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 988/1963] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:49.214368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.225535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84030c2-1e7a-4f73-9b45-d28a414c5dbd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.231102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.231725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.232013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.232290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.232974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.235140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.246984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 976/1964] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:49.251268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.262622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc5031-eb8e-466a-855b-abf921e15b6d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.268132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3f809-4bd2-4884-ab1f-c0894b8b987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.268786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f809-4bd2-4884-ab1f-c0894b8b987e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.269233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f809-4bd2-4884-ab1f-c0894b8b987e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.269567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f809-4bd2-4884-ab1f-c0894b8b987e admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.270380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f809-4bd2-4884-ab1f-c0894b8b987e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.271999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f809-4bd2-4884-ab1f-c0894b8b987e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.285904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 989/1965] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:49.291083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.303211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f164f4-5612-4e00-9e02-68adf6ce1aef admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.308346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.309156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.309535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.309883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] PATH_INFO: `/v3/projects/41ffd22f90fb4f7380598fd6aaaf1685/users/35127169bcdc439c95ad4d0cd65f0a58/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.313412 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:49.333945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ffd22f90fb4f7380598fd6aaaf1685, user_id=35127169bcdc439c95ad4d0cd65f0a58, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.337373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.356115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 977/1966] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:49 2022] PUT /identity/v3/projects/41ffd22f90fb4f7380598fd6aaaf1685/users/35127169bcdc439c95ad4d0cd65f0a58/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:49.367129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.367420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.367965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.408993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] MFA Rules not processed for user `35127169bcdc439c95ad4d0cd65f0a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:49.410342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:49.475156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:49.496303 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 990/1967] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:49.501069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.583567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015182ed-8c7d-47cb-bf01-77c53245ec31 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.590309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.590968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.591254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.591529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.592181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.594176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.645512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 978/1968] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:49.649305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.663434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cdac9-0da5-4b93-a000-448dec33359a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.745736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.746490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.747074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.747527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.749778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.751821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.777195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 991/1969] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:49.780870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.792499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af35082-a216-4557-88b5-49809d93e6fd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.798254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.798894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.799203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.799478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.800132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.802890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.814065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 979/1970] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:49.818449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.832389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5921ac-4cbb-49eb-89c8-99b89b49dc03 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.838512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.839208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.839491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.839774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] PATH_INFO: `/v3/projects/af4d96fdadd04180b23a1d26558e6ead/users/f1d781b902674b458a2dd04aa46cdf7b/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.857279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4d96fdadd04180b23a1d26558e6ead, user_id=f1d781b902674b458a2dd04aa46cdf7b, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.860029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.875036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 992/1971] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:49 2022] PUT /identity/v3/projects/af4d96fdadd04180b23a1d26558e6ead/users/f1d781b902674b458a2dd04aa46cdf7b/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:49.879932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.911888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e36d5-ea87-46f6-8e52-d8dcdc2cc28e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.985130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90e58c-0305-49a3-ac55-ef82f0a1af76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.985794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90e58c-0305-49a3-ac55-ef82f0a1af76 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.986076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90e58c-0305-49a3-ac55-ef82f0a1af76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.986351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90e58c-0305-49a3-ac55-ef82f0a1af76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.986999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90e58c-0305-49a3-ac55-ef82f0a1af76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.988591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90e58c-0305-49a3-ac55-ef82f0a1af76 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.001424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 980/1972] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:50.005653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.016571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04706-29d7-4854-916a-59d21e9db579 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.021576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.024507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.024994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.025529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] PATH_INFO: `/v3/projects/af4d96fdadd04180b23a1d26558e6ead/users/f1d781b902674b458a2dd04aa46cdf7b/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.027896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4d96fdadd04180b23a1d26558e6ead, user_id=f1d781b902674b458a2dd04aa46cdf7b, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.030551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.041818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 993/1973] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:50 2022] PUT /identity/v3/projects/af4d96fdadd04180b23a1d26558e6ead/users/f1d781b902674b458a2dd04aa46cdf7b/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:50.052947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.053161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.053472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.082815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] MFA Rules not processed for user `f1d781b902674b458a2dd04aa46cdf7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:50.084068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:50.128230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:50.156269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 981/1974] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:50.167709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.169434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:50.228875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1348dc-7c54-48eb-bd78-4e6afd21ef54 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.232591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf1271-c951-43fc-95e7-84cc6cf36afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.233200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf1271-c951-43fc-95e7-84cc6cf36afc service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.233496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf1271-c951-43fc-95e7-84cc6cf36afc service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.233781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf1271-c951-43fc-95e7-84cc6cf36afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.294896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:50.297897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf1271-c951-43fc-95e7-84cc6cf36afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.299751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf1271-c951-43fc-95e7-84cc6cf36afc service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.312701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 994/1975] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:53.661174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:53.662619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:53.737450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53560e80-bf47-4948-8360-5b3a7a3a15e5 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:53.742559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:53.743203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:53.743516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:53.743785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] PATH_INFO: `/v3/users/9979f803da8040dfa9ba4bfca12a22a5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:53.744913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9979f803da8040dfa9ba4bfca12a22a5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:53.746928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:53.794092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9979f803da8040dfa9ba4bfca12a22a5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:53.801880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9979f803da8040dfa9ba4bfca12a22a5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:53.802508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9979f803da8040dfa9ba4bfca12a22a5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:53.810506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9979f803da8040dfa9ba4bfca12a22a5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:53.822598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9979f803da8040dfa9ba4bfca12a22a5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:53.828392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 982/1976] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:53 2022] DELETE /identity/v3/users/9979f803da8040dfa9ba4bfca12a22a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:53.832767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:53.833220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:53.833307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:53.852996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:53.854226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:53.912452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 995/1977] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:53.922351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:53.923766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:53.980772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c0e4f-a432-4a77-90e7-c9b732b7cbbb admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:53.984347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:53.984949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:53.985245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:53.985532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:53.994423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:53.995832 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:54.000679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:54.005857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 983/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:54.270490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:54.271926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:54.285189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630f805-dd95-46e5-95a2-cc58a49d3f21 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:54.291091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:54.292187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:54.292765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:54.293418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] PATH_INFO: `/v3/projects/e8e0c2743a3c4495a0d9a1edecbf534b` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:54.294612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e0c2743a3c4495a0d9a1edecbf534b)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:54.296555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:54.345608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e0c2743a3c4495a0d9a1edecbf534b', 'request_id': 'req-e5a32416-9233-434f-b81c-5b967465adbf'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.354253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e0c2743a3c4495a0d9a1edecbf534b', 'request_id': 'req-e5a32416-9233-434f-b81c-5b967465adbf'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.367068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e0c2743a3c4495a0d9a1edecbf534b', 'request_id': 'req-e5a32416-9233-434f-b81c-5b967465adbf'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.378895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] The token cache is being invalidate because project e8e0c2743a3c4495a0d9a1edecbf534b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:54.379271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.380189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.381830 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 996/1979] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:54 2022] DELETE /identity/v3/projects/e8e0c2743a3c4495a0d9a1edecbf534b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:54.387710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:54.404595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99093f50-4951-4b0e-937d-dff0ab21edc2 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:54.498101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:54.498749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:54.499033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:54.499309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] PATH_INFO: `/v3/users/0d5620eea9074ecaa4b8fa46b9b4d47b` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:54.500433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d5620eea9074ecaa4b8fa46b9b4d47b)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:54.502447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:54.576312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d5620eea9074ecaa4b8fa46b9b4d47b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.583538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d5620eea9074ecaa4b8fa46b9b4d47b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.584223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d5620eea9074ecaa4b8fa46b9b4d47b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.591211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d5620eea9074ecaa4b8fa46b9b4d47b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.603345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d5620eea9074ecaa4b8fa46b9b4d47b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:54.609184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 984/1980] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:54 2022] DELETE /identity/v3/users/0d5620eea9074ecaa4b8fa46b9b4d47b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:54.883995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:54.886650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:54.904572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a32416-9233-434f-b81c-5b967465adbf admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:54.962652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:54.963216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:54.963497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:54.963773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] PATH_INFO: `/v3/projects/ee82814e1e5141b1b08c095fbff51d3c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:54.964884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee82814e1e5141b1b08c095fbff51d3c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:54.966848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:54.990875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:55.015385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee82814e1e5141b1b08c095fbff51d3c', 'request_id': 'req-71070540-5ff3-411d-b281-6acc6fd84ef5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.021563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee82814e1e5141b1b08c095fbff51d3c', 'request_id': 'req-71070540-5ff3-411d-b281-6acc6fd84ef5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.029138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee82814e1e5141b1b08c095fbff51d3c', 'request_id': 'req-71070540-5ff3-411d-b281-6acc6fd84ef5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.040541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] The token cache is being invalidate because project ee82814e1e5141b1b08c095fbff51d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:55.040838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.041519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.043042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 997/1981] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:54 2022] DELETE /identity/v3/projects/ee82814e1e5141b1b08c095fbff51d3c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:55.047361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.049061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.062986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78185dcb-6b0f-44b0-98df-ad699a524332 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.119065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.119700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.119985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.120266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] PATH_INFO: `/v3/users/8b7caca316ab4a13bf75d6f5b0e538e5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.121385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7caca316ab4a13bf75d6f5b0e538e5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.123346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.145996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:55.172068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7caca316ab4a13bf75d6f5b0e538e5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.179846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7caca316ab4a13bf75d6f5b0e538e5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.180393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7caca316ab4a13bf75d6f5b0e538e5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.187555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7caca316ab4a13bf75d6f5b0e538e5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.199369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ddac0f5b-068c-42fc-8478-f4cf09b39ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7caca316ab4a13bf75d6f5b0e538e5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.205162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 985/1982] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:55 2022] DELETE /identity/v3/users/8b7caca316ab4a13bf75d6f5b0e538e5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:55.571519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.573059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.587359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070540-5ff3-411d-b281-6acc6fd84ef5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.654728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.655370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.655644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.655916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] PATH_INFO: `/v3/projects/58367b2c2e9048b1b684794c393997cf` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.657021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=58367b2c2e9048b1b684794c393997cf)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.658995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.708169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58367b2c2e9048b1b684794c393997cf', 'request_id': 'req-2da14280-efc3-403c-a41b-82f9a465f7aa'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.715073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58367b2c2e9048b1b684794c393997cf', 'request_id': 'req-2da14280-efc3-403c-a41b-82f9a465f7aa'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.723511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58367b2c2e9048b1b684794c393997cf', 'request_id': 'req-2da14280-efc3-403c-a41b-82f9a465f7aa'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.734698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] The token cache is being invalidate because project 58367b2c2e9048b1b684794c393997cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:55.734984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.735621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2da14280-efc3-403c-a41b-82f9a465f7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.737097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 998/1983] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:55 2022] DELETE /identity/v3/projects/58367b2c2e9048b1b684794c393997cf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:55.743671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.743927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.744196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.764372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:55.765619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.824992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 986/1984] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:55.833150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.833433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.833755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.855207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:55.856432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.864430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 999/1985] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:55.869290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.879213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6096ad6-7aaa-4efd-8221-8d5fa813ad98 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.884173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.884784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.885065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.885338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.886022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:55.886322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.887905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.900688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 987/1986] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:55.905089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.915346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611631be-732b-4f86-b6f0-72c4f5256fd9 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.919914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.920511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.920790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.921063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.921720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.923619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.952875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1000/1987] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:55 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:55.957434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.970962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f23f9e-c2a4-4bee-a134-e616ff1f82ca admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.027986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.028598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.028882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.029158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.029825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.031743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.053029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 988/1988] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:55 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:56.058101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.068224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb732599-cf2f-49f0-b05d-0a44ce0248b3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.073163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.073812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.074098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.074374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.075049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.076642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.082112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:56.091331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1001/1989] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:56.092729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.102919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ec245-c104-4e76-9063-6fa4b25655e8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.107517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799d771-e94d-4d98-bf5b-a086cbbf19b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.108118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799d771-e94d-4d98-bf5b-a086cbbf19b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.108403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799d771-e94d-4d98-bf5b-a086cbbf19b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.108683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799d771-e94d-4d98-bf5b-a086cbbf19b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.109329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799d771-e94d-4d98-bf5b-a086cbbf19b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.110960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799d771-e94d-4d98-bf5b-a086cbbf19b0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.122312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 989/1990] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:56.126499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.136377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c5580-cc4f-4dc0-9cec-cadf5a8aade5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.140897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.141572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.141854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.142125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] PATH_INFO: `/v3/projects/7d3d097b175e41e1ac3d2eaf38442f05/users/5681ea448a734bc69bd1305c08bd71b3/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.158201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3d097b175e41e1ac3d2eaf38442f05, user_id=5681ea448a734bc69bd1305c08bd71b3, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.160857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.174291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1002/1991] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:56 2022] PUT /identity/v3/projects/7d3d097b175e41e1ac3d2eaf38442f05/users/5681ea448a734bc69bd1305c08bd71b3/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:56.186459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.186833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.187189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.195298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:56.217138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] MFA Rules not processed for user `5681ea448a734bc69bd1305c08bd71b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:56.218384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:56.286756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 990/1992] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:56.291309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.363716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefefc4-a0ff-45d5-b253-aeb687a03a24 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.371221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.372201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.372666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.373082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.373984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.376782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.416393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1003/1993] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:56 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:56.421639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.452350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897fb91-59fc-4d15-b27a-03429ef7daf4 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.527777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.528402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.529380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.529642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.530572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.532590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.559219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 991/1994] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:56 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:56.563861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.575538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42782d0b-e264-4fe2-864e-0518041ef229 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.580640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.581318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.581626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.581897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.582535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.584265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.595808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1004/1995] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:56.600239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.618300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703f773-a0da-44e8-a6a3-1a8aeca30575 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.623579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.624518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.624973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.625392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] PATH_INFO: `/v3/projects/ab4e88f380464971ac81c6626e9d1a30/users/131cf7608d61420cb889ae136deb75be/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.645055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4e88f380464971ac81c6626e9d1a30, user_id=131cf7608d61420cb889ae136deb75be, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.649101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.667452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 992/1996] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:56 2022] PUT /identity/v3/projects/ab4e88f380464971ac81c6626e9d1a30/users/131cf7608d61420cb889ae136deb75be/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:00:56.671997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.683221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acd4d3-572a-474e-9a27-872b83f88316 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.745012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a28023-00ed-47ea-9e58-e9c580b0fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.745787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a28023-00ed-47ea-9e58-e9c580b0fbad admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.746140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a28023-00ed-47ea-9e58-e9c580b0fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.746482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a28023-00ed-47ea-9e58-e9c580b0fbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.747268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a28023-00ed-47ea-9e58-e9c580b0fbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.749348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a28023-00ed-47ea-9e58-e9c580b0fbad admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.767143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1005/1997] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:56.770875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:56.784524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34dca3-0d3a-487e-90d0-149b6561a67e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:56.796606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:56.797856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.798148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.798432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] PATH_INFO: `/v3/projects/ab4e88f380464971ac81c6626e9d1a30/users/131cf7608d61420cb889ae136deb75be/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.800771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4e88f380464971ac81c6626e9d1a30, user_id=131cf7608d61420cb889ae136deb75be, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:56.805973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:56.822513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 993/1998] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:56 2022] PUT /identity/v3/projects/ab4e88f380464971ac81c6626e9d1a30/users/131cf7608d61420cb889ae136deb75be/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:56.831878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:56.832469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:56.832855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:56.868285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] MFA Rules not processed for user `131cf7608d61420cb889ae136deb75be`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:56.870136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:56.972375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:57.014518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1006/1999] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:00:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:57.027003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:57.028927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:57.083720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:57.090493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb6477-325d-4417-8664-fa4364ad8939 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:57.094864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:57.095543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:57.095867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:57.096178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:57.107256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:57.108694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:57.119004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 994/2000] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:57.222542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:57.235696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22913-d4c9-4af7-84f0-8dfc49736a9e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:57.239428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:57.240037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:57.240312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:57.240583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:57.300740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:57.302316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:57.311258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1007/2001] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:59.172588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.182421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48415a-fff5-4600-8620-12336b2098ad service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.186818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.187435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.187719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.187993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.189957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:59.278898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.280340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e38329-5531-46a9-bc9a-9d0da8e498e3 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.294564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 995/2002] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:00.013558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.084175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c021ef-87ef-43d7-9d59-c40e315387cf service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.090353 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.091024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.091304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.091575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] PATH_INFO: `/v3/users/5681ea448a734bc69bd1305c08bd71b3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.092797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5681ea448a734bc69bd1305c08bd71b3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.095042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.157156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5681ea448a734bc69bd1305c08bd71b3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.168729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5681ea448a734bc69bd1305c08bd71b3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.170355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5681ea448a734bc69bd1305c08bd71b3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.170355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5681ea448a734bc69bd1305c08bd71b3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.178482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5681ea448a734bc69bd1305c08bd71b3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.187142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:00.204672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1008/2003] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:00 2022] DELETE /identity/v3/users/5681ea448a734bc69bd1305c08bd71b3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:00.206785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.207281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.207281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.231128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:00.232381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:00.299215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 996/2004] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:00.309844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.311571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:00.375807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b870cc-be53-4fea-b5af-ab0f54818a34 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.381091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.381754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.382166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.383501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.394254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.395694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.407731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1009/2005] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:00.799207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.800681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:00.827698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4d81-98fa-4304-b951-ac931cad2795 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.836545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.837214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.837507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.837784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] PATH_INFO: `/v3/projects/7d3d097b175e41e1ac3d2eaf38442f05` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.838973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3d097b175e41e1ac3d2eaf38442f05)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.840904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.967364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3d097b175e41e1ac3d2eaf38442f05', 'request_id': 'req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.976044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3d097b175e41e1ac3d2eaf38442f05', 'request_id': 'req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.995567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3d097b175e41e1ac3d2eaf38442f05', 'request_id': 'req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.014580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] The token cache is being invalidate because project 7d3d097b175e41e1ac3d2eaf38442f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:01.015184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.015967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.020922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 997/2006] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:00 2022] DELETE /identity/v3/projects/7d3d097b175e41e1ac3d2eaf38442f05 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:01.022170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:01.040793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc471d-ff90-47a9-bb3b-b2d6d3cc9ebe service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:01.126450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:01.127440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.127872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.128321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] PATH_INFO: `/v3/users/131cf7608d61420cb889ae136deb75be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.131143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=131cf7608d61420cb889ae136deb75be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:01.134618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:01.204636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131cf7608d61420cb889ae136deb75be'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.220393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131cf7608d61420cb889ae136deb75be'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.221249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131cf7608d61420cb889ae136deb75be'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.222022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131cf7608d61420cb889ae136deb75be'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.236342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1282768a-760f-476c-8fe3-c40c9cb04550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131cf7608d61420cb889ae136deb75be'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.256493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1010/2007] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:01 2022] DELETE /identity/v3/users/131cf7608d61420cb889ae136deb75be => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:01.559400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:01.560922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:01.574191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2decc-d730-47d2-8a33-d9a4bb6ebd5c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:01.580200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:01.647521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:01.649416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.649742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.650114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] PATH_INFO: `/v3/projects/ab4e88f380464971ac81c6626e9d1a30` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.651313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4e88f380464971ac81c6626e9d1a30)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:01.653268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:01.707505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4e88f380464971ac81c6626e9d1a30', 'request_id': 'req-e856a1f8-5634-4829-95f4-363160e2769c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.715267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4e88f380464971ac81c6626e9d1a30', 'request_id': 'req-e856a1f8-5634-4829-95f4-363160e2769c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.722885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4e88f380464971ac81c6626e9d1a30', 'request_id': 'req-e856a1f8-5634-4829-95f4-363160e2769c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.734692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] The token cache is being invalidate because project ab4e88f380464971ac81c6626e9d1a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:01.735018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.735681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e856a1f8-5634-4829-95f4-363160e2769c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.737199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 998/2008] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:01 2022] DELETE /identity/v3/projects/ab4e88f380464971ac81c6626e9d1a30 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:01.742357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.742646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.746718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.768351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:01.769589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:01.775908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:01.838705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1011/2009] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:01.849303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.849673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.849972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.873077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:01.875078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:01.886331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 999/2010] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:01.891517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:01.905853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752611b-44f0-452a-81c5-4e5207100cdb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:01.911945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:01.913281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.913430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.913631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.914558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:01.914970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:01.916643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:01.930803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1012/2011] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:01.935529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:01.947238 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f26cd-86b5-4b15-8a14-50da1d2aff5b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:01.952874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:01.953531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.953820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.954092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.954909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:01.957852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:01.999852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1000/2012] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:01 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:02.001216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:02.019362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3fbe3-d6a3-4a6b-9ede-f3fb9bbd3d8d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:02.079354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:02.080059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.080243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.080514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.081143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:02.083184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:02.116734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1013/2013] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:02 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:02.118425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:02.129677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b7286-2ae1-4624-b16b-96be43191fc7 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:02.134993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:02.135604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.135883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.136157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.136799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:02.138429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:02.149648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1001/2014] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:02.153714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:02.168385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4522b2c-922f-43b8-9554-cd6927b69162 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:02.173178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d54f2d-9082-4405-b850-1443a2f70507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:02.173845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d54f2d-9082-4405-b850-1443a2f70507 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.174127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d54f2d-9082-4405-b850-1443a2f70507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.174397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d54f2d-9082-4405-b850-1443a2f70507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.175064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d54f2d-9082-4405-b850-1443a2f70507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:02.176661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d54f2d-9082-4405-b850-1443a2f70507 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:02.188754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1014/2015] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:02.192879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:02.203852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6afea8-f350-4a9a-94b7-a768ca228e01 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:02.208724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:02.209383 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.209707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.210036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] PATH_INFO: `/v3/projects/5889240dc3e64e798ac6baa04836731a/users/d070ffadd77c4fe0baaf20c627ce57e7/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.227227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5889240dc3e64e798ac6baa04836731a, user_id=d070ffadd77c4fe0baaf20c627ce57e7, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:02.230160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:02.244719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1002/2016] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:02 2022] PUT /identity/v3/projects/5889240dc3e64e798ac6baa04836731a/users/d070ffadd77c4fe0baaf20c627ce57e7/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:02.255324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.255641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.255881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.286419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] MFA Rules not processed for user `d070ffadd77c4fe0baaf20c627ce57e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:02.287645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:02.358372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1015/2017] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:02.369321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:02.371046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:02.406618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:02.431165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d793ed2-bcd4-4823-bbce-e3093607137b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:02.434831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edf02d-2830-4877-ad1a-6e1868d081f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:02.435447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edf02d-2830-4877-ad1a-6e1868d081f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.435733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edf02d-2830-4877-ad1a-6e1868d081f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.436011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edf02d-2830-4877-ad1a-6e1868d081f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.444385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edf02d-2830-4877-ad1a-6e1868d081f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:02.445962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edf02d-2830-4877-ad1a-6e1868d081f5 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:02.454892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1003/2018] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:12.019789 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:12.021190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:12.031583 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:12.090816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6d456-157a-4b22-bc90-849a7aadfc3a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:12.095364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:12.095948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:12.096230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:12.096501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] PATH_INFO: `/v3/users/35127169bcdc439c95ad4d0cd65f0a58` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:12.097640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35127169bcdc439c95ad4d0cd65f0a58)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:12.099573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:12.149771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35127169bcdc439c95ad4d0cd65f0a58'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.157072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35127169bcdc439c95ad4d0cd65f0a58'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.157723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35127169bcdc439c95ad4d0cd65f0a58'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.158323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35127169bcdc439c95ad4d0cd65f0a58'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.165967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35127169bcdc439c95ad4d0cd65f0a58'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.183175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1016/2019] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:12 2022] DELETE /identity/v3/users/35127169bcdc439c95ad4d0cd65f0a58 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:12.187752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:12.188363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:12.188363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:12.208469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:12.209731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:12.278844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1004/2020] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:12.289885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:12.291823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:12.356744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75790edb-5174-46a2-b494-0bdc1565a810 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:12.360714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:12.361325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:12.361636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:12.361911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:12.371505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:12.373117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:12.384191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1017/2021] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:12.659217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:12.660650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:12.673475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede6ff-7c03-41ff-81b0-c024a2ce6aa3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:12.678139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:12.678779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:12.679066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:12.679952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] PATH_INFO: `/v3/projects/41ffd22f90fb4f7380598fd6aaaf1685` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:12.681571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ffd22f90fb4f7380598fd6aaaf1685)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:12.683500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:12.751653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ffd22f90fb4f7380598fd6aaaf1685', 'request_id': 'req-59d11555-3445-4265-98c9-a0c16130b399'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.761412 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ffd22f90fb4f7380598fd6aaaf1685', 'request_id': 'req-59d11555-3445-4265-98c9-a0c16130b399'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.770759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ffd22f90fb4f7380598fd6aaaf1685', 'request_id': 'req-59d11555-3445-4265-98c9-a0c16130b399'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.784266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] The token cache is being invalidate because project 41ffd22f90fb4f7380598fd6aaaf1685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:12.784565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.785277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.786908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1005/2022] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:12 2022] DELETE /identity/v3/projects/41ffd22f90fb4f7380598fd6aaaf1685 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:12.791094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:12.802336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eff7a-6c16-487a-ae2f-095119115a8f service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:12.832438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:12.867725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:12.868480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:12.868759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:12.869034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] PATH_INFO: `/v3/users/f1d781b902674b458a2dd04aa46cdf7b` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:12.870192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d781b902674b458a2dd04aa46cdf7b)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:12.872162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:12.920763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d781b902674b458a2dd04aa46cdf7b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.928059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d781b902674b458a2dd04aa46cdf7b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.928820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d781b902674b458a2dd04aa46cdf7b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.930569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d781b902674b458a2dd04aa46cdf7b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.937670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3b9fdb7f-a773-4da2-aed0-be3bbe98e4e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d781b902674b458a2dd04aa46cdf7b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:12.955068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1018/2023] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:12 2022] DELETE /identity/v3/users/f1d781b902674b458a2dd04aa46cdf7b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:12.997249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:12.998726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.011961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d11555-3445-4265-98c9-a0c16130b399 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.082882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.083704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.084094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.084512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] PATH_INFO: `/v3/users/d070ffadd77c4fe0baaf20c627ce57e7` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.086149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d070ffadd77c4fe0baaf20c627ce57e7)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.088843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:13.090920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:13.158180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd070ffadd77c4fe0baaf20c627ce57e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.170029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd070ffadd77c4fe0baaf20c627ce57e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.170668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd070ffadd77c4fe0baaf20c627ce57e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.181564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd070ffadd77c4fe0baaf20c627ce57e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.198014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd070ffadd77c4fe0baaf20c627ce57e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.207490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1006/2024] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:12 2022] DELETE /identity/v3/users/d070ffadd77c4fe0baaf20c627ce57e7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:13.212886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.214097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.214542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.248569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:13.249912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.267048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:13.268852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.360001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1019/2025] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:13.371395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:13.372917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.389541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f8894-adee-4ee0-b7f6-ccdf7ede0343 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.395156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.395936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.396221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.396496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] PATH_INFO: `/v3/projects/af4d96fdadd04180b23a1d26558e6ead` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.400190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4d96fdadd04180b23a1d26558e6ead)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.400550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:13.466501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:13.470724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3481f9-38d9-4226-a0b4-85c37bae285c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.474467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4d96fdadd04180b23a1d26558e6ead', 'request_id': 'req-77f9176a-deb2-4402-b945-56608ff6cb71'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.484250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4d96fdadd04180b23a1d26558e6ead', 'request_id': 'req-77f9176a-deb2-4402-b945-56608ff6cb71'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.494164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4d96fdadd04180b23a1d26558e6ead', 'request_id': 'req-77f9176a-deb2-4402-b945-56608ff6cb71'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.508953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] The token cache is being invalidate because project af4d96fdadd04180b23a1d26558e6ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:13.509364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.510862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-77f9176a-deb2-4402-b945-56608ff6cb71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:13.512834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1007/2026] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:13 2022] DELETE /identity/v3/projects/af4d96fdadd04180b23a1d26558e6ead => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:13.518361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1fb59-91ff-4a2c-bda5-4aa7ff6e6052 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.518654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1fb59-91ff-4a2c-bda5-4aa7ff6e6052 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.518934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1fb59-91ff-4a2c-bda5-4aa7ff6e6052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.534588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.535273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.535641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.535987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.538179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.541680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-3ed1fb59-91ff-4a2c-bda5-4aa7ff6e6052 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:13.542929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3ed1fb59-91ff-4a2c-bda5-4aa7ff6e6052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.551193 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:13.614866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1008/2027] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:13.624390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.624792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.625082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.626692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.628299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:13.640751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1020/2028] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:13.648664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:13.649940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.659202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1009/2029] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:13.666006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:13.679577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77afa0-5060-474b-8480-2830659f9091 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.686357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.687423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.688157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.688536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.689527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:13.689903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.692237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:13.719834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1021/2030] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:13.724720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:13.736194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfe166-8221-460f-9269-1554f487ffb3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.743923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.744164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.744533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.744875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.746224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.748304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:13.792755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1010/2031] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:13 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:13.797154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:13.812132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc0470-b93c-4355-bc0b-ee1fee0239de admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.932638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.933717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.934626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.935007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.935845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.938137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:13.990932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1022/2032] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:13 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:13.995424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.006889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44ac49-2a2f-466a-abf2-b6326243fee7 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.012539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.013160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.013549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.013802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.014455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.016100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.031412 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1011/2033] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:14.032363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.047199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e03a0a-6376-4a0a-8941-9d68055bb97a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.052662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6707-58e3-4ffa-9a77-49d4cf79a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.053287 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6707-58e3-4ffa-9a77-49d4cf79a524 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.053606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6707-58e3-4ffa-9a77-49d4cf79a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.053881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6707-58e3-4ffa-9a77-49d4cf79a524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.055068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6707-58e3-4ffa-9a77-49d4cf79a524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.056682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6707-58e3-4ffa-9a77-49d4cf79a524 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.072642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1023/2034] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:14.076744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.089910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc505eee-a7a8-47f2-b18f-4f8cb1162460 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.094931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.095614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.096071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.096357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] PATH_INFO: `/v3/projects/e43f7a864fad4d659dbb7c785355fb5c/users/8a58cfa61d3041d3ae55265667b612f0/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.114191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43f7a864fad4d659dbb7c785355fb5c, user_id=8a58cfa61d3041d3ae55265667b612f0, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.117653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.138095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1012/2035] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:14 2022] PUT /identity/v3/projects/e43f7a864fad4d659dbb7c785355fb5c/users/8a58cfa61d3041d3ae55265667b612f0/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:14.150521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.150914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.151292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.158525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.160219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.185939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] MFA Rules not processed for user `8a58cfa61d3041d3ae55265667b612f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:14.187078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.196460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:14.245940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01743-3f25-4cd9-986f-fc22961da42a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.251665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.252474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.252602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.252853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] PATH_INFO: `/v3/projects/5889240dc3e64e798ac6baa04836731a` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.254182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5889240dc3e64e798ac6baa04836731a)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.256267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.276180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1024/2036] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:14.284799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.286401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.310448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:14.312343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5889240dc3e64e798ac6baa04836731a', 'request_id': 'req-52686aae-52ac-49b3-858f-d41732cddc26'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.320083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5889240dc3e64e798ac6baa04836731a', 'request_id': 'req-52686aae-52ac-49b3-858f-d41732cddc26'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.329682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5889240dc3e64e798ac6baa04836731a', 'request_id': 'req-52686aae-52ac-49b3-858f-d41732cddc26'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.343721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] The token cache is being invalidate because project 5889240dc3e64e798ac6baa04836731a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:14.344139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.344897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-52686aae-52ac-49b3-858f-d41732cddc26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.346781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1013/2037] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:14 2022] DELETE /identity/v3/projects/5889240dc3e64e798ac6baa04836731a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:14.351371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafe9cf-3219-48ed-9b74-546face16b02 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.351792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafe9cf-3219-48ed-9b74-546face16b02 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.352198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafe9cf-3219-48ed-9b74-546face16b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.374273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-baafe9cf-3219-48ed-9b74-546face16b02 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:14.375702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-baafe9cf-3219-48ed-9b74-546face16b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.380099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5840167-9c91-411d-9f5e-166f2924db5e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.431985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.432811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.433127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.433404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.435347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.440200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1014/2038] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:14.448662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.448996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.449272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.469162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:14.470677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.480870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1015/2039] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:14.485868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.496405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4fe1d-350d-4e37-ac5c-091f1aef57ec None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.501036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.501361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.501679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.501958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.502231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.502838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.502945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:14.503298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.504892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.507593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:14.519103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1016/2040] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:14.520002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1025/2041] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:14.523860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.536024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a2386-62b4-4e1c-b1d4-588786a8c26e service glance] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.546663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.547362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.547738 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.548077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.548838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.551772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.589549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1026/2042] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:14.593519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.606122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb9541-159d-45e6-8e6a-a18fdb8d8577 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.631139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.633914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.679369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.680851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.681197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.681586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.686008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.688885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.727842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1017/2043] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:14.732488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.751630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672431cd-1efa-4a76-861f-08fc54f53e1e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.756291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49668-c714-45f9-8bf4-ff6a3d21ccd5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.758876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.759164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.759579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.759941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.761553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.762902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.763452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.765203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.765845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.766225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] PATH_INFO: `/v3/users/9829ff4e240343c1b2681820ccf06c7c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.767542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9829ff4e240343c1b2681820ccf06c7c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.770366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.778866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1018/2044] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:14.782829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.795754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96efa66-f268-4292-8e4c-70a4ad60e8a4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.801641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.802528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.802986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.803653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.804970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.806644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.836880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1019/2045] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:14.838989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.852934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2367-7b34-4312-84e4-9d63bd34b8a8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.859452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bf892-c592-436e-8ca7-4d6262ceec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.861996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bf892-c592-436e-8ca7-4d6262ceec5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.862430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bf892-c592-436e-8ca7-4d6262ceec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.863261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bf892-c592-436e-8ca7-4d6262ceec5d admin admin] PATH_INFO: `/v3/projects/83b78140663c4691a6d4d76bd1286ebe/users/24751c1a00f34c228ed8aa67e6845c1f/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.873770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9829ff4e240343c1b2681820ccf06c7c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.887182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9829ff4e240343c1b2681820ccf06c7c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.888317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9829ff4e240343c1b2681820ccf06c7c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.889313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9829ff4e240343c1b2681820ccf06c7c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.890301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bf892-c592-436e-8ca7-4d6262ceec5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b78140663c4691a6d4d76bd1286ebe, user_id=24751c1a00f34c228ed8aa67e6845c1f, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.893013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bf892-c592-436e-8ca7-4d6262ceec5d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.897764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02ec5457-fe71-498a-ae59-a5a5ac448945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9829ff4e240343c1b2681820ccf06c7c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.911336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:14.917369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1020/2046] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:14 2022] PUT /identity/v3/projects/83b78140663c4691a6d4d76bd1286ebe/users/24751c1a00f34c228ed8aa67e6845c1f/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:14.928490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1027/2047] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:14 2022] DELETE /identity/v3/users/9829ff4e240343c1b2681820ccf06c7c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:14.929396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.929756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.930045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.932752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.933107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.933451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.957953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:14.959519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] MFA Rules not processed for user `24751c1a00f34c228ed8aa67e6845c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:14.959831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:14.961301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.020618 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:15.045591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1021/2048] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:15.049736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1028/2049] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:15.050816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.052621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.053561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.056002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.068932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106fec4-1c0a-45a0-896c-04ba5a7ab261 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.075070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.076009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.076467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.076987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.077817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.080226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.119502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1022/2050] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:15.125037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.162625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d8730-09e0-47f5-bf8b-96d9704f538c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.221236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046cc09-1711-42ea-ac01-022665c4a1de admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.229306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.230109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.230496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.230869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.232580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.233423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.234337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.234765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.235756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.237944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.243637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.245530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.257391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1029/2051] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:15.264406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1023/2052] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:15.268115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.279708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59514ae-b48a-4e03-9a45-12f4b074d285 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.285078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.285750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.286306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.286306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.286922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.289079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.300684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1030/2053] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:15.305277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.315712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723d9fe-e48c-45cc-9407-2842995e30d1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.322154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c8861-08b8-4c07-9dec-e1c1d1a3eed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.323408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c8861-08b8-4c07-9dec-e1c1d1a3eed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.323752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c8861-08b8-4c07-9dec-e1c1d1a3eed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.324078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c8861-08b8-4c07-9dec-e1c1d1a3eed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.324871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c8861-08b8-4c07-9dec-e1c1d1a3eed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.326819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c8861-08b8-4c07-9dec-e1c1d1a3eed1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.340771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1024/2054] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:15.344746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.356284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee21c-831c-4516-8b08-1f0b12c3de24 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.361492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.361970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.362279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.362557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] PATH_INFO: `/v3/projects/10b83981afd9412d8fd56138c4ab5485/users/a5086eb5fb9245af888936ef28e48ff3/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.381855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b83981afd9412d8fd56138c4ab5485, user_id=a5086eb5fb9245af888936ef28e48ff3, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.386111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.404271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1031/2055] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:15 2022] PUT /identity/v3/projects/10b83981afd9412d8fd56138c4ab5485/users/a5086eb5fb9245af888936ef28e48ff3/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:15.416754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.417043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.417331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.453054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] MFA Rules not processed for user `a5086eb5fb9245af888936ef28e48ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:15.454620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.497829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:15.538925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1025/2056] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:15.543261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.610417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfafc5-b972-4d34-8108-0dd5ed43288c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.615430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.616036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.616314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.616638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.617449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.619375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.654793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1032/2057] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:15.659243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.668028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.669767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.673257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8962a-4c43-47c7-8554-afa521283c94 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.679325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:15.742462 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94e510-191a-46b7-abf1-e4b9794be2e7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.745454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.746445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.746921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.747746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.748067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.748379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.748853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.749358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.749491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] PATH_INFO: `/v3/projects/ff8220c0fada4f9696f946260903492e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.750952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8220c0fada4f9696f946260903492e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.751440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.753413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.782940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1026/2058] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:15.787042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.798733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a162c0-3742-4778-a937-e95351a11df5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.803852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.804499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.804789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.805059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.805747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.807372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.816542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8220c0fada4f9696f946260903492e', 'request_id': 'req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.817728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1027/2059] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:15.822094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.823857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8220c0fada4f9696f946260903492e', 'request_id': 'req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.832544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8220c0fada4f9696f946260903492e', 'request_id': 'req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.834548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612edc2b-cdb9-486f-a60e-e84ee90c5e6c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.845384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] The token cache is being invalidate because project ff8220c0fada4f9696f946260903492e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:15.845784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.846509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-577461d9-87d6-4af5-8ca5-4860a1c6b4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.848089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1033/2060] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:15 2022] DELETE /identity/v3/projects/ff8220c0fada4f9696f946260903492e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:15.853779 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d656267-296e-43f2-9e3d-43aeb90800e2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.854203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d656267-296e-43f2-9e3d-43aeb90800e2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.854747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d656267-296e-43f2-9e3d-43aeb90800e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.881622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8d656267-296e-43f2-9e3d-43aeb90800e2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:15.883337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8d656267-296e-43f2-9e3d-43aeb90800e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.917236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.918157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.918502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.918848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] PATH_INFO: `/v3/projects/83ce260ebfe44a17bb1c8264fdd87e36/users/2647c5ec80fa4e0a9bda4986d5480bb5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.939316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ce260ebfe44a17bb1c8264fdd87e36, user_id=2647c5ec80fa4e0a9bda4986d5480bb5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.942164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.961532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1028/2061] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:15 2022] PUT /identity/v3/projects/83ce260ebfe44a17bb1c8264fdd87e36/users/2647c5ec80fa4e0a9bda4986d5480bb5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:15.966498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.969095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.973904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1034/2062] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:15.987338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.987627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.987908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.011065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:16.012378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.016772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:16.052005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2cbd7-ef9b-4a9a-9f36-757f4e39e5aa admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.061376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:16.068195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.068833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.069115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.069390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.070233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.072516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.087833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1029/2063] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:16.088912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.100296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d7cb-fd9b-4c9a-ad9d-05baa98afaf5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.104792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1035/2064] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:16.109121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.110912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49f881-f835-428c-9379-a8cc8304018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.111876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49f881-f835-428c-9379-a8cc8304018a admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.112155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49f881-f835-428c-9379-a8cc8304018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.112437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49f881-f835-428c-9379-a8cc8304018a admin admin] PATH_INFO: `/v3/projects/83ce260ebfe44a17bb1c8264fdd87e36/users/2647c5ec80fa4e0a9bda4986d5480bb5/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.120524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49f881-f835-428c-9379-a8cc8304018a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ce260ebfe44a17bb1c8264fdd87e36, user_id=2647c5ec80fa4e0a9bda4986d5480bb5, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.123239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49f881-f835-428c-9379-a8cc8304018a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.124153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940c2fd-99f1-4f47-aa36-fe2a3fd5c7cb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.129913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.130598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.130885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.131356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.131981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:16.132341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.134011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.137512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1030/2065] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:16 2022] PUT /identity/v3/projects/83ce260ebfe44a17bb1c8264fdd87e36/users/2647c5ec80fa4e0a9bda4986d5480bb5/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:16.146553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.146849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.147127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.152913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1036/2066] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:16.153888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.166389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f38cff-5eed-4dc7-8b1f-7e24d2f859eb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.193619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] MFA Rules not processed for user `2647c5ec80fa4e0a9bda4986d5480bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:16.196604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.267347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.268307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.268731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.269137 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.269901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.272238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.306399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1031/2067] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:16.307126 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1037/2068] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:16 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:16.313650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.321120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.323030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.324876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86dcd8-875e-42a8-9235-d9fce2c2f6ef None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.384038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb206cd-dc2c-427b-ba24-7b969d013b67 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.385740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.386335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.386643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.386911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.387656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.387756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.388302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.388580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.388852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.389602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.390625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.414450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1032/2069] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:16 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:16.418455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.428337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb09705-d4df-49f2-b38a-fdbb225821e4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.441306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.441948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.442227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.442501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.443156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.445946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.458803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1033/2070] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:16.463129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.464975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.466610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.473440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5837b-4a96-4c7e-97b7-25875a9c367c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.476249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1038/2071] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:16.480760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3938ec1-f9c8-484c-b8e1-c6c6944aae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.481366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938ec1-f9c8-484c-b8e1-c6c6944aae56 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.481704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938ec1-f9c8-484c-b8e1-c6c6944aae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.481946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938ec1-f9c8-484c-b8e1-c6c6944aae56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.482589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938ec1-f9c8-484c-b8e1-c6c6944aae56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.484247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938ec1-f9c8-484c-b8e1-c6c6944aae56 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.496072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1034/2072] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:16.500131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.507856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:16.512370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3647f-d7f0-448a-8b29-e01f8d0149ff service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.521157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.521898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.522178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.522447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] PATH_INFO: `/v3/projects/011dd202e9ce47968630123fd813b0e3/users/adb63fcdb36f4b66a2d380c4414eb278/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.547972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=011dd202e9ce47968630123fd813b0e3, user_id=adb63fcdb36f4b66a2d380c4414eb278, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.552426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.577566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1039/2073] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:16 2022] PUT /identity/v3/projects/011dd202e9ce47968630123fd813b0e3/users/adb63fcdb36f4b66a2d380c4414eb278/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:16.580845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.581135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.581416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.613054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:16.617775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] MFA Rules not processed for user `adb63fcdb36f4b66a2d380c4414eb278`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:16.619674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.689500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1035/2074] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:16.702016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.703933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.767610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038a18-ac97-49d4-b49c-7196b0f85f5d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.772351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.773322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.774159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.774159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.785158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.786863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.806186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1040/2075] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:18.157683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:18.168859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2e012-9685-43c5-86d3-9891120db30e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:18.173842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:18.174470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:18.174749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:18.175021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:18.176839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:18.245856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:18.247769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:18.265190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1036/2076] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:18.984968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:18.993751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3cde1-b7e9-437d-9b7a-b22dba89da43 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:18.997311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e83b7-1471-4faf-b05f-6f17aba5842b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:18.997963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e83b7-1471-4faf-b05f-6f17aba5842b service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:18.998240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e83b7-1471-4faf-b05f-6f17aba5842b service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:18.998511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e83b7-1471-4faf-b05f-6f17aba5842b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.079207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e83b7-1471-4faf-b05f-6f17aba5842b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:19.080879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e83b7-1471-4faf-b05f-6f17aba5842b service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:19.091314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1041/2077] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:19.462180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:19.541394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98176e-5c02-48c0-afc1-d0a45ad896fa service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:19.546875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:19.547524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.547806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.548080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] PATH_INFO: `/v3/users/adb63fcdb36f4b66a2d380c4414eb278` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.549229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb63fcdb36f4b66a2d380c4414eb278)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:19.551357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:19.611516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:19.616293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb63fcdb36f4b66a2d380c4414eb278'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.627090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb63fcdb36f4b66a2d380c4414eb278'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.627742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb63fcdb36f4b66a2d380c4414eb278'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.637337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb63fcdb36f4b66a2d380c4414eb278'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.668229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb63fcdb36f4b66a2d380c4414eb278'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.683563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.683852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.684070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.685535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1037/2078] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:19 2022] DELETE /identity/v3/users/adb63fcdb36f4b66a2d380c4414eb278 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:19.706337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:19.707566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:19.777504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1042/2079] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:19.788098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:19.790024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:19.867138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3561f97-e225-4f4d-ba91-a1b17b28f088 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:19.871275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951294a-bfb5-4314-96cb-f0d27d584025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:19.871902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951294a-bfb5-4314-96cb-f0d27d584025 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.872214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951294a-bfb5-4314-96cb-f0d27d584025 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.872574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951294a-bfb5-4314-96cb-f0d27d584025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.884717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951294a-bfb5-4314-96cb-f0d27d584025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:19.886210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951294a-bfb5-4314-96cb-f0d27d584025 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:19.900199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1038/2080] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:20.284028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.285579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:20.319769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e19ec-efe5-4814-b3cc-418be9a70ab0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.333108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.335995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.337987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.338504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] PATH_INFO: `/v3/projects/011dd202e9ce47968630123fd813b0e3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.343533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] RBAC: Authorizing `identity:delete_project(project_id=011dd202e9ce47968630123fd813b0e3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:20.348234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:20.380860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:20.437698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011dd202e9ce47968630123fd813b0e3', 'request_id': 'req-2aa08c7f-31f9-40f1-83f1-30dc9390216b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.445884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011dd202e9ce47968630123fd813b0e3', 'request_id': 'req-2aa08c7f-31f9-40f1-83f1-30dc9390216b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.474849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011dd202e9ce47968630123fd813b0e3', 'request_id': 'req-2aa08c7f-31f9-40f1-83f1-30dc9390216b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.497209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] The token cache is being invalidate because project 011dd202e9ce47968630123fd813b0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:20.497626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.498271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2aa08c7f-31f9-40f1-83f1-30dc9390216b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.516856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1043/2081] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:20 2022] DELETE /identity/v3/projects/011dd202e9ce47968630123fd813b0e3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:20.522462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.524932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.525424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.552432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:20.554028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:20.633697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1039/2082] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:20.640645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.640931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.641222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.688773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:20.690783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:20.716724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1044/2083] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:20.718788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.733583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7101f-b748-4447-94c1-8d20935b7b9c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.741307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.743331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.744257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.745143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.746190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:20.746630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:20.748783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:20.766279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1040/2084] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:20.775388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.787729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27362485-0787-4eb4-a1dc-40a9c771ab8c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.794076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.794421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.794701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.794977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.795611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:20.797519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:20.833499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1045/2085] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:20 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:20.835993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.852765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38634bf7-e324-487e-be61-653f4846761d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.910380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:20.930345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.931489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.931889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.932393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.934574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:20.936839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:20.969701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1041/2086] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:20 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:20.973855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.987212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0c23-15b4-4384-9152-4417075811a8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.996061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.996878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.997140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.997401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.998484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.000033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.017488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1046/2087] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:21.021839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.040213 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8926f63-e51c-4615-999c-69fe285d9d3b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.049214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b3bfd-5652-4346-af03-7b3d0caadf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.051336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3bfd-5652-4346-af03-7b3d0caadf2f admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.052116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3bfd-5652-4346-af03-7b3d0caadf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.053262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3bfd-5652-4346-af03-7b3d0caadf2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.053702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b3bfd-5652-4346-af03-7b3d0caadf2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.059961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b3bfd-5652-4346-af03-7b3d0caadf2f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.075128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1042/2088] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:21.080075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.096938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeea2f-d2de-42a9-90fb-e12c854d74fb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.105846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.108525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.108847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.109127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] PATH_INFO: `/v3/projects/8fc4cb21c64e4180ad7005f8f81c16e7/users/12c816acf1504e5699789092394932a5/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.135366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc4cb21c64e4180ad7005f8f81c16e7, user_id=12c816acf1504e5699789092394932a5, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.140685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.158635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1047/2089] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:21 2022] PUT /identity/v3/projects/8fc4cb21c64e4180ad7005f8f81c16e7/users/12c816acf1504e5699789092394932a5/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:21.173693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.173793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.174045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.209031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] MFA Rules not processed for user `12c816acf1504e5699789092394932a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:21.210680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.296854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1043/2090] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:21.310005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.311956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.364947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.367185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.400306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131cc2b-d477-48c9-b683-1bdecdfa09a6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.404009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.404622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.404903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.405174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.413862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.415293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.425114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1048/2091] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:21.449112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc432734-6391-4c50-bb5e-61010bdce8f7 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.452449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.458675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.459545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.460096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.460961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] PATH_INFO: `/v3/users/12f441d491f846018630b0871734aab0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.462273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f441d491f846018630b0871734aab0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.464290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.469557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068517e-8101-4a75-be34-bc4bb6f29570 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.474935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.475558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.475871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.476149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] PATH_INFO: `/v3/users/12c816acf1504e5699789092394932a5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.477455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c816acf1504e5699789092394932a5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.479405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.513717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f441d491f846018630b0871734aab0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.518675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:21.521720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f441d491f846018630b0871734aab0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.522601 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f441d491f846018630b0871734aab0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.530650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:21.532545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c816acf1504e5699789092394932a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.532955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f441d491f846018630b0871734aab0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.555872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c816acf1504e5699789092394932a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.556397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-15b77365-b82d-4940-9951-f26fc03ae8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f441d491f846018630b0871734aab0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.556604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c816acf1504e5699789092394932a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.557557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c816acf1504e5699789092394932a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.566990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1044/2092] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:21 2022] DELETE /identity/v3/users/12f441d491f846018630b0871734aab0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:21.574291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c816acf1504e5699789092394932a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.596822 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1049/2093] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:21 2022] DELETE /identity/v3/users/12c816acf1504e5699789092394932a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:21.601846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.602744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.603233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.627695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:21.629024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.707008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1045/2094] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:21.723476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.727129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.797262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.799514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.820410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6c36-6b86-405c-ad17-a580c46650ce None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.825252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.827857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e18ab8-b853-47d2-834a-a063300e19d8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.832250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8aaa95-162b-420f-b004-4a1d8605bdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.832866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aaa95-162b-420f-b004-4a1d8605bdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.833036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.833147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aaa95-162b-420f-b004-4a1d8605bdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.833373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.833491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aaa95-162b-420f-b004-4a1d8605bdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.833689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] PATH_INFO: `/v3/projects/97165e035a134b5c9178f531801a533c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.834924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97165e035a134b5c9178f531801a533c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.838402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.845264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8aaa95-162b-420f-b004-4a1d8605bdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.847556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8aaa95-162b-420f-b004-4a1d8605bdf6 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.861167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1050/2095] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:21.915039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97165e035a134b5c9178f531801a533c', 'request_id': 'req-4254e692-be85-49a8-8a2e-ef5a11732f91'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.922804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97165e035a134b5c9178f531801a533c', 'request_id': 'req-4254e692-be85-49a8-8a2e-ef5a11732f91'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.943941 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97165e035a134b5c9178f531801a533c', 'request_id': 'req-4254e692-be85-49a8-8a2e-ef5a11732f91'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.956896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] The token cache is being invalidate because project 97165e035a134b5c9178f531801a533c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:21.957225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.958186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4254e692-be85-49a8-8a2e-ef5a11732f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.964382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1046/2096] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:21 2022] DELETE /identity/v3/projects/97165e035a134b5c9178f531801a533c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:21.965843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.966115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.966395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.991113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:21.993020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.079183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1051/2097] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:22.088533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.089163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.090275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.127464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:22.129080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.138176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1047/2098] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:22.142414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.153530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a789-80e5-44fc-a932-3b29e4cae3c4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.158918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.159950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.160283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.160682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.161447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:22.161852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.164464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.180460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1052/2099] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:22.185271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.196321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce2f9-2f85-4336-8f9c-1502d99322fc None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.200866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.202747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.203180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.203363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.203650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.203929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.204575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.206547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.220178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b49384-07c5-4bde-a196-db1400285312 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.226855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.227739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.228058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.228365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] PATH_INFO: `/v3/projects/8fc4cb21c64e4180ad7005f8f81c16e7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.230226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc4cb21c64e4180ad7005f8f81c16e7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.232557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.242663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1048/2100] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:22.243676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.275463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:22.280657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:22.286458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc4cb21c64e4180ad7005f8f81c16e7', 'request_id': 'req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:22.293739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc4cb21c64e4180ad7005f8f81c16e7', 'request_id': 'req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:22.301260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc4cb21c64e4180ad7005f8f81c16e7', 'request_id': 'req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:22.312762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ad7b-2332-454b-afff-01ff3254b1a6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.313056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] The token cache is being invalidate because project 8fc4cb21c64e4180ad7005f8f81c16e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:22.313517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:22.314003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc31a8b0-ea73-4e23-83b2-7e9fb0dfc34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:22.315521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1053/2101] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:22 2022] DELETE /identity/v3/projects/8fc4cb21c64e4180ad7005f8f81c16e7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:22.320872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f6363-c018-4bc1-a38f-20967eb7a4a9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.321159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f6363-c018-4bc1-a38f-20967eb7a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.321435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f6363-c018-4bc1-a38f-20967eb7a4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.341028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e94f6363-c018-4bc1-a38f-20967eb7a4a9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:22.342258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e94f6363-c018-4bc1-a38f-20967eb7a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.372438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.373086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.373367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.373671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.374302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.376997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.400937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1049/2102] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:22.405332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.407469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.414342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1054/2103] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:22.420851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d939-6b69-4f35-89d2-219135c0e543 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.422875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.423166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.423439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.425658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.426262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.426540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.426819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.427460 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.429571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.440186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1050/2104] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:22.444077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:22.444498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.446002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.454305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1055/2105] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:22.455793 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eafe85-be21-48a3-8e7e-95466ce27237 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.458746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.461647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.462245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.462658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.462960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.463599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.465258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.469170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d506bfb-e2e3-4c3f-a046-a2ebe9134ecd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.473905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.474496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.474778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.475054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.475817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:22.476108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.476612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1051/2106] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:22.477996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.480856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.490545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc02b7-b2f0-4148-8cef-4fa076a51b7b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.493768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1056/2107] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:01:22.496196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e20a9b-99a5-4c56-a01c-3e868b947fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.496850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20a9b-99a5-4c56-a01c-3e868b947fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.496974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.497195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20a9b-99a5-4c56-a01c-3e868b947fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.497493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20a9b-99a5-4c56-a01c-3e868b947fac admin admin] PATH_INFO: `/v3/projects/966032a30afd43088cc5a5e9808534a5/users/8423f205300c47c8bcce162c46430628/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.508264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffe567-5aa8-4d22-8edd-dba58c6c8308 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.513084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.514114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.514234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e20a9b-99a5-4c56-a01c-3e868b947fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=966032a30afd43088cc5a5e9808534a5, user_id=8423f205300c47c8bcce162c46430628, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.514774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.515048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.515810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.516923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e20a9b-99a5-4c56-a01c-3e868b947fac admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.518014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.531119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1052/2108] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:22 2022] PUT /identity/v3/projects/966032a30afd43088cc5a5e9808534a5/users/8423f205300c47c8bcce162c46430628/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:22.541548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.541869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.542277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.550842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1057/2109] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:22.555707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.578391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] MFA Rules not processed for user `8423f205300c47c8bcce162c46430628`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:22.579731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.624917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdec29b-8077-4476-afdd-95e501dd91c2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.640000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.640000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.640000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.640000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.640632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.640632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.661497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1058/2110] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:22.663887 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.668212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1053/2111] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:22.672555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.674255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b1608-61a9-4c90-b618-98ef7e55e32c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.678851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.679743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.680149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.680421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.681058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.682716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.683289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9176ef9-23b7-4a5d-837b-a6e209a87314 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.688735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.689334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.689654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.689928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.690559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.692494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.692676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1059/2112] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:22.697154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.702015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:22.705286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:22.709710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345defc-acd0-49d4-b3f3-3e31fc34f5ed admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.716566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.717406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.717735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.718008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.718640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.720264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.723309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1054/2113] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:22.727703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.732812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1060/2114] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:22.736922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.751891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cd44b-3c79-4c7f-b412-92d69aca1a91 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.797632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2113b6-bc9d-4f02-b6b8-07858b5a3214 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.803619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.804517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.804804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.805123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.805806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.808054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.815577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415aeba9-3e87-490d-a3c6-0a73b812625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.816599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415aeba9-3e87-490d-a3c6-0a73b812625e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.817432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415aeba9-3e87-490d-a3c6-0a73b812625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.817890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415aeba9-3e87-490d-a3c6-0a73b812625e admin admin] PATH_INFO: `/v3/projects/8d9e4d501d0a4849b1b7707f31542783/users/dc517a3b178242fc8982bc0fe616ed83/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.834211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1055/2115] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:22.838750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.839209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415aeba9-3e87-490d-a3c6-0a73b812625e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9e4d501d0a4849b1b7707f31542783, user_id=dc517a3b178242fc8982bc0fe616ed83, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.842324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415aeba9-3e87-490d-a3c6-0a73b812625e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.850081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92324fab-aefb-4d2c-bf43-d2cfff87c6e0 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.857718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1061/2116] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:22 2022] PUT /identity/v3/projects/8d9e4d501d0a4849b1b7707f31542783/users/dc517a3b178242fc8982bc0fe616ed83/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:22.868986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.869274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.869964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.912942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] MFA Rules not processed for user `dc517a3b178242fc8982bc0fe616ed83`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:22.914289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.935604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.936199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.936814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.937216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.937916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.939826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.955610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1056/2117] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:22.959794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.982418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0afd81-5029-4947-84c3-37f0c87274fc admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.987422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.987972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.988258 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.988535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] PATH_INFO: `/v3/projects/be64f3b6f8224c548b516fb715f3e70e/users/4f04a4e3932147eba38fa5ef6ae9a53c/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.997207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1062/2118] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:23.008768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=be64f3b6f8224c548b516fb715f3e70e, user_id=4f04a4e3932147eba38fa5ef6ae9a53c, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.009424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.010831 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.011721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.025537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1057/2119] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:22 2022] PUT /identity/v3/projects/be64f3b6f8224c548b516fb715f3e70e/users/4f04a4e3932147eba38fa5ef6ae9a53c/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:23.029334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.073204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e780a-3d85-42d3-9ca5-d898da580396 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.088961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:23.089304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:23.102098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73206-b2ad-408e-8f3f-200fe65401ad admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.107102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.108487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.108773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.109047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.109743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.111393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.123432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.123606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1058/2120] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:23.124044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.124322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.124742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.127709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.137889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f92fb-5547-4719-ad8e-877bd629d320 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.142517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d9a2f-5c56-4fe6-a818-e371e3e83666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.143156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d9a2f-5c56-4fe6-a818-e371e3e83666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.143436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d9a2f-5c56-4fe6-a818-e371e3e83666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.143712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d9a2f-5c56-4fe6-a818-e371e3e83666 admin admin] PATH_INFO: `/v3/projects/be64f3b6f8224c548b516fb715f3e70e/users/4f04a4e3932147eba38fa5ef6ae9a53c/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.145729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d9a2f-5c56-4fe6-a818-e371e3e83666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be64f3b6f8224c548b516fb715f3e70e, user_id=4f04a4e3932147eba38fa5ef6ae9a53c, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.149378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d9a2f-5c56-4fe6-a818-e371e3e83666 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.164770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1059/2121] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:23 2022] PUT /identity/v3/projects/be64f3b6f8224c548b516fb715f3e70e/users/4f04a4e3932147eba38fa5ef6ae9a53c/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:23.176288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605faee-b248-4807-a8c2-33902b35644c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.176656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605faee-b248-4807-a8c2-33902b35644c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.177007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605faee-b248-4807-a8c2-33902b35644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.192732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.195481 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.209979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1063/2122] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:23.215048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a605faee-b248-4807-a8c2-33902b35644c None None] MFA Rules not processed for user `4f04a4e3932147eba38fa5ef6ae9a53c`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:23.216256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a605faee-b248-4807-a8c2-33902b35644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.273223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.274959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.289233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8e19f-3315-48a7-a2a9-ab46dc785286 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.307225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1060/2123] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:23.319348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605faee-b248-4807-a8c2-33902b35644c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.321406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a605faee-b248-4807-a8c2-33902b35644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.368001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.368654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.368938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.369219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] PATH_INFO: `/v3/users/24751c1a00f34c228ed8aa67e6845c1f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.370873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24751c1a00f34c228ed8aa67e6845c1f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.372541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.386384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605faee-b248-4807-a8c2-33902b35644c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.390926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022e70a-531d-4297-98cb-d642c91c213b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.391539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022e70a-531d-4297-98cb-d642c91c213b service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.391819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022e70a-531d-4297-98cb-d642c91c213b service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.392085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022e70a-531d-4297-98cb-d642c91c213b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.401860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022e70a-531d-4297-98cb-d642c91c213b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.403449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022e70a-531d-4297-98cb-d642c91c213b service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.415086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1061/2124] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:23.444136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24751c1a00f34c228ed8aa67e6845c1f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:23.458814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24751c1a00f34c228ed8aa67e6845c1f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:23.459696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24751c1a00f34c228ed8aa67e6845c1f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:23.460680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24751c1a00f34c228ed8aa67e6845c1f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:23.470064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24751c1a00f34c228ed8aa67e6845c1f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:23.482604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:23.499672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1064/2125] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:23 2022] DELETE /identity/v3/users/24751c1a00f34c228ed8aa67e6845c1f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:23.504157 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.504561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.504910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.530989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:23.532918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.617815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1062/2126] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:23.629536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.631022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.712957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c1af8-8da8-400c-aabe-3dd3eb1e3374 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.736100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a524a48-07c3-4593-8368-73897bd4743b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.736919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a524a48-07c3-4593-8368-73897bd4743b service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.737366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a524a48-07c3-4593-8368-73897bd4743b service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.743164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a524a48-07c3-4593-8368-73897bd4743b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.759328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a524a48-07c3-4593-8368-73897bd4743b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.761082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a524a48-07c3-4593-8368-73897bd4743b service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.772341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1065/2127] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:23.948983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.950504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.959283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:23.964481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809629a-8551-4b5b-b1c8-6e66af78e5ff None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.974169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.974892 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.975180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.975456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] PATH_INFO: `/v3/users/8423f205300c47c8bcce162c46430628` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.977403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8423f205300c47c8bcce162c46430628)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.979437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:24.035577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8423f205300c47c8bcce162c46430628'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.045162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8423f205300c47c8bcce162c46430628'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.045891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8423f205300c47c8bcce162c46430628'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.053711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8423f205300c47c8bcce162c46430628'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.068422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8423f205300c47c8bcce162c46430628'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.075162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1063/2128] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:23 2022] DELETE /identity/v3/users/8423f205300c47c8bcce162c46430628 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:24.079352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4be620-6c6a-4624-af18-735795557504 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:24.079809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4be620-6c6a-4624-af18-735795557504 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:24.080082 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4be620-6c6a-4624-af18-735795557504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:24.099845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cd4be620-6c6a-4624-af18-735795557504 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:24.101590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cd4be620-6c6a-4624-af18-735795557504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.164854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1066/2129] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:24.175440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.181617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4be620-6c6a-4624-af18-735795557504 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.182812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.183307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cd4be620-6c6a-4624-af18-735795557504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.197740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4be620-6c6a-4624-af18-735795557504 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:24.203713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:24.204430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:24.204783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:24.205163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] PATH_INFO: `/v3/projects/83b78140663c4691a6d4d76bd1286ebe` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:24.206537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b78140663c4691a6d4d76bd1286ebe)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:24.208705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:24.264163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b78140663c4691a6d4d76bd1286ebe', 'request_id': 'req-2bc7c67d-5e14-41af-83d3-d512c6fe9932'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.270643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b78140663c4691a6d4d76bd1286ebe', 'request_id': 'req-2bc7c67d-5e14-41af-83d3-d512c6fe9932'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.275996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e2f36-2b59-4558-95c6-9f0c145a3194 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:24.280020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b78140663c4691a6d4d76bd1286ebe', 'request_id': 'req-2bc7c67d-5e14-41af-83d3-d512c6fe9932'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.290843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] The token cache is being invalidate because project 83b78140663c4691a6d4d76bd1286ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:24.291164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.291820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.293355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1067/2130] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:24 2022] DELETE /identity/v3/projects/83b78140663c4691a6d4d76bd1286ebe => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:24.297284 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.298615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.323431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:24.328232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:24.329067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:24.329372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:24.329637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:24.332437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.371672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c67d-5e14-41af-83d3-d512c6fe9932 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:24.376256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:24.376892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:24.377176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:24.377450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] PATH_INFO: `/v3/users/a5086eb5fb9245af888936ef28e48ff3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:24.378602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5086eb5fb9245af888936ef28e48ff3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:24.380555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:24.381347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:24.402711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:24.404274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:24.419406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1064/2131] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:24.433544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5086eb5fb9245af888936ef28e48ff3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.441492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5086eb5fb9245af888936ef28e48ff3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.442233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5086eb5fb9245af888936ef28e48ff3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.442908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5086eb5fb9245af888936ef28e48ff3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.450717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5086eb5fb9245af888936ef28e48ff3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.470344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1068/2132] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:24 2022] DELETE /identity/v3/users/a5086eb5fb9245af888936ef28e48ff3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:24.706809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.712759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.732987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a27b90-24bf-4d29-8b76-cd0f17847a44 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:24.771685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.774247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.813790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:24.814512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:24.814913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:24.815282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] PATH_INFO: `/v3/projects/966032a30afd43088cc5a5e9808534a5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:24.816813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966032a30afd43088cc5a5e9808534a5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:24.820291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:24.865649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19128a8-8629-47d9-a72d-7fc21daeecc1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:24.866389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:24.867532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:24.867827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:24.868100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] PATH_INFO: `/v3/projects/10b83981afd9412d8fd56138c4ab5485` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:24.869448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b83981afd9412d8fd56138c4ab5485)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:24.871370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:24.888924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966032a30afd43088cc5a5e9808534a5', 'request_id': 'req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.895673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966032a30afd43088cc5a5e9808534a5', 'request_id': 'req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.905807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966032a30afd43088cc5a5e9808534a5', 'request_id': 'req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.918861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] The token cache is being invalidate because project 966032a30afd43088cc5a5e9808534a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:24.919187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.919842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.921378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1065/2133] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:24 2022] DELETE /identity/v3/projects/966032a30afd43088cc5a5e9808534a5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:24.925447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.926866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:24.940901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:24.950663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b83981afd9412d8fd56138c4ab5485', 'request_id': 'req-e86dd528-e33c-40ae-9b9e-90a183bdd169'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.959156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b83981afd9412d8fd56138c4ab5485', 'request_id': 'req-e86dd528-e33c-40ae-9b9e-90a183bdd169'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.968705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b83981afd9412d8fd56138c4ab5485', 'request_id': 'req-e86dd528-e33c-40ae-9b9e-90a183bdd169'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.985296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] The token cache is being invalidate because project 10b83981afd9412d8fd56138c4ab5485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:24.985894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.986579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:24.988691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1069/2134] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:24 2022] DELETE /identity/v3/projects/10b83981afd9412d8fd56138c4ab5485 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:24.992705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:24.994758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:25.010931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50dae6-7b61-41f8-a5cb-c57d81d269f1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:25.031470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:25.076473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dd528-e33c-40ae-9b9e-90a183bdd169 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:25.092288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:25.092940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.093214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.093513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] PATH_INFO: `/v3/users/2647c5ec80fa4e0a9bda4986d5480bb5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.094640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2647c5ec80fa4e0a9bda4986d5480bb5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:25.097651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:25.112169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:25.112805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.113088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.113363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] PATH_INFO: `/v3/users/4f04a4e3932147eba38fa5ef6ae9a53c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.114670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f04a4e3932147eba38fa5ef6ae9a53c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:25.117348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:25.159056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2647c5ec80fa4e0a9bda4986d5480bb5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.167055 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2647c5ec80fa4e0a9bda4986d5480bb5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.167743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2647c5ec80fa4e0a9bda4986d5480bb5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.168448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2647c5ec80fa4e0a9bda4986d5480bb5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.170857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f04a4e3932147eba38fa5ef6ae9a53c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.176876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2647c5ec80fa4e0a9bda4986d5480bb5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.178079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f04a4e3932147eba38fa5ef6ae9a53c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.178720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f04a4e3932147eba38fa5ef6ae9a53c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.187508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f04a4e3932147eba38fa5ef6ae9a53c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.196295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1070/2135] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:24 2022] DELETE /identity/v3/users/2647c5ec80fa4e0a9bda4986d5480bb5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:25.200588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f04a4e3932147eba38fa5ef6ae9a53c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.206920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1066/2136] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:24 2022] DELETE /identity/v3/users/4f04a4e3932147eba38fa5ef6ae9a53c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:25.524960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:25.526609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:25.581564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:25.605552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10499340-c8d7-4b5e-aeb1-e20ed4382c27 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:25.617384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:25.619162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:25.682853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:25.683416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.683694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.684002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] PATH_INFO: `/v3/projects/83ce260ebfe44a17bb1c8264fdd87e36` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.685102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ce260ebfe44a17bb1c8264fdd87e36)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:25.687100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:25.704011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5bd6f-d6a2-465e-89f8-90cdec97db2e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:25.708469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:25.709101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.709380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.709685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] PATH_INFO: `/v3/projects/be64f3b6f8224c548b516fb715f3e70e` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.710933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be64f3b6f8224c548b516fb715f3e70e)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:25.712837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:25.742460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ce260ebfe44a17bb1c8264fdd87e36', 'request_id': 'req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.749748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ce260ebfe44a17bb1c8264fdd87e36', 'request_id': 'req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.758206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ce260ebfe44a17bb1c8264fdd87e36', 'request_id': 'req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.763786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:25.770412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] The token cache is being invalidate because project 83ce260ebfe44a17bb1c8264fdd87e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:25.770706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.771423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e4e33dfc-1c79-447f-9c6c-c97b3e3523b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.772119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be64f3b6f8224c548b516fb715f3e70e', 'request_id': 'req-b852ffcf-91be-4937-ad71-ef19742bf525'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.773256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1071/2137] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:25 2022] DELETE /identity/v3/projects/83ce260ebfe44a17bb1c8264fdd87e36 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:25.780222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be64f3b6f8224c548b516fb715f3e70e', 'request_id': 'req-b852ffcf-91be-4937-ad71-ef19742bf525'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.790013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be64f3b6f8224c548b516fb715f3e70e', 'request_id': 'req-b852ffcf-91be-4937-ad71-ef19742bf525'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.802226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] The token cache is being invalidate because project be64f3b6f8224c548b516fb715f3e70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:25.802581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.803261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b852ffcf-91be-4937-ad71-ef19742bf525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:25.805415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1067/2138] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:25 2022] DELETE /identity/v3/projects/be64f3b6f8224c548b516fb715f3e70e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:25.810582 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.810868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.811147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.832450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:25.833709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:25.921325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1072/2139] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:25.929853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.930229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.930521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.954447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:25.955610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:25.964289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1068/2140] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:25.968624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:25.979389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c426b-3f3f-47b8-847a-4479db4b4dab None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:25.984045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:25.984666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.984946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.985220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.986012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:25.986203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:25.987811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.000425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1073/2141] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:26.004900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.014487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6b6e-17d1-4d87-9abd-395654fcfaf7 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.019192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.019842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.020081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.020363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.020996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.023527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.056080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1069/2142] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:26.060316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.070336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158ffc4-9851-4bc9-9187-c99bd80891bb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.079098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:26.130503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.131104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.131389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.131666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.132322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.134274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.159328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1074/2143] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:26.160484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.171474 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c4013-fbd7-471b-945e-d502eb784753 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.176132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.176732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.177017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.177294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.177968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.179722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.189767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1070/2144] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:26.193391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.203409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ae7f-b1da-4a20-9c81-ebc8ec74a8cd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.208059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f36ba-18cd-426a-a6e5-396b15d0be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.208592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f36ba-18cd-426a-a6e5-396b15d0be21 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.208873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f36ba-18cd-426a-a6e5-396b15d0be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.209145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f36ba-18cd-426a-a6e5-396b15d0be21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.209808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f36ba-18cd-426a-a6e5-396b15d0be21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.212151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f36ba-18cd-426a-a6e5-396b15d0be21 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.222579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1075/2145] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:26.227280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.237152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003cb8d-0434-489a-8104-2fe654e7b43c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.242565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.243212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.243489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.243759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] PATH_INFO: `/v3/projects/2a22890362174d059ac21917991ef68b/users/87013d98ad33486098f785b70fd4e990/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.259044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a22890362174d059ac21917991ef68b, user_id=87013d98ad33486098f785b70fd4e990, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.261727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.274590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1071/2146] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:26 2022] PUT /identity/v3/projects/2a22890362174d059ac21917991ef68b/users/87013d98ad33486098f785b70fd4e990/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:26.284501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.284791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.285072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.313653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] MFA Rules not processed for user `87013d98ad33486098f785b70fd4e990`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:26.314871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:26.378484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1076/2147] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:26.383623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.451233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d28824-5558-401e-b768-81eef10b48c6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.455991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.456595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.456875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.457146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.457839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.459810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.486328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1072/2148] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:26.490568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.502389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d161f30-bde5-4286-9a65-9fcc89aa59ec None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.569088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.570840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.571129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.571411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.572050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.573996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.599615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1077/2149] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:26.601170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.612179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee1887-b0c8-4f95-bd21-b2e24dc72cdf admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.617041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.617689 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.617978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.618384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.618976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.620746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.634296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1073/2150] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:26.638369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.651509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b11c-2219-433d-ac5a-b23532aa2054 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.656227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.656875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.657158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.657433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] PATH_INFO: `/v3/projects/6defb039ba3548949c94fad3ee678ad1/users/b176841e89b24932830fbcd69135d497/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.668223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:26.676188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6defb039ba3548949c94fad3ee678ad1, user_id=b176841e89b24932830fbcd69135d497, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.679643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.693215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1078/2151] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:26 2022] PUT /identity/v3/projects/6defb039ba3548949c94fad3ee678ad1/users/b176841e89b24932830fbcd69135d497/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:26.698418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.709135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e239c5-31a6-4160-b616-fe1e3cad2803 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.756724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:26.778549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4442edf-de61-4489-b90b-bf00479995c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.779209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442edf-de61-4489-b90b-bf00479995c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.779569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442edf-de61-4489-b90b-bf00479995c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.779900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442edf-de61-4489-b90b-bf00479995c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.780629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4442edf-de61-4489-b90b-bf00479995c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.782433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4442edf-de61-4489-b90b-bf00479995c1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.793240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1074/2152] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:26.797606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.808739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0d54-817c-48e2-875a-f948f8c5ac9d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.813325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.813999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.814282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.814555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] PATH_INFO: `/v3/projects/6defb039ba3548949c94fad3ee678ad1/users/b176841e89b24932830fbcd69135d497/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.816475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6defb039ba3548949c94fad3ee678ad1, user_id=b176841e89b24932830fbcd69135d497, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.819164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.830276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1079/2153] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:26 2022] PUT /identity/v3/projects/6defb039ba3548949c94fad3ee678ad1/users/b176841e89b24932830fbcd69135d497/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:26.841930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.842257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.842558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.870533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] MFA Rules not processed for user `b176841e89b24932830fbcd69135d497`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:26.871752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:26.944106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1075/2154] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:26.955810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.957580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:27.042656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9f5a4-d541-4719-b517-513e2f599360 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:27.046452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:27.047078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:27.047358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:27.047637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:27.105529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:27.106991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:27.116044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1080/2155] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:33.618220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:33.619650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:33.703700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18a4aa-b828-4207-9256-e24c54835f5f None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:33.709041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:33.709724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:33.710009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:33.710311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] PATH_INFO: `/v3/users/dc517a3b178242fc8982bc0fe616ed83` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:33.711484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc517a3b178242fc8982bc0fe616ed83)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:33.714275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:33.784387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc517a3b178242fc8982bc0fe616ed83'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:33.793715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc517a3b178242fc8982bc0fe616ed83'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:33.794417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc517a3b178242fc8982bc0fe616ed83'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:33.801618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc517a3b178242fc8982bc0fe616ed83'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:33.803001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:33.817076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-26029514-1fc9-4d17-91db-5a5d494964b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc517a3b178242fc8982bc0fe616ed83'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:33.821435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:33.822966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:33.823659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1076/2156] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:33 2022] DELETE /identity/v3/users/dc517a3b178242fc8982bc0fe616ed83 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:33.827625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:33.827910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:33.828186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:33.834360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95469a73-2003-483a-ab9c-f5594cb59860 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:33.847245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:33.848456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:33.892283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:33.892938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:33.893215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:33.893514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:33.895246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:33.919508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1077/2157] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:33.928940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:33.930378 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:33.940666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:33.961838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:33.963286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:33.972815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1081/2158] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:33.989086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb67b3-b18c-4832-b642-a9749bb5b75c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:33.992710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:33.994118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:33.994495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:33.994780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:34.003957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:34.005372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:34.016501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1078/2159] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:34.326516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:34.329930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:34.361743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeb682-8e6e-4dc8-aa7c-77cb804e9eb5 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:34.366314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:34.367193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:34.367597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:34.368565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] PATH_INFO: `/v3/projects/8d9e4d501d0a4849b1b7707f31542783` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:34.370365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9e4d501d0a4849b1b7707f31542783)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:34.372290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:34.419929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9e4d501d0a4849b1b7707f31542783', 'request_id': 'req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:34.426304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9e4d501d0a4849b1b7707f31542783', 'request_id': 'req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:34.434536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9e4d501d0a4849b1b7707f31542783', 'request_id': 'req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:34.444928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] The token cache is being invalidate because project 8d9e4d501d0a4849b1b7707f31542783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:34.445221 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:34.445861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-12b0e14e-b1fb-4016-a68a-caa19a15b1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:34.448067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1082/2160] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:34 2022] DELETE /identity/v3/projects/8d9e4d501d0a4849b1b7707f31542783 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:44.012106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=60,keystone:DELETE=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:44.441864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:49.447621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.449034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.462112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50197233-5210-4aaa-99cf-0ba87b8551db service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.517217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.517867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.518145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.518418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] PATH_INFO: `/v3/users/8a58cfa61d3041d3ae55265667b612f0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.519505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a58cfa61d3041d3ae55265667b612f0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.521418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.577249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a58cfa61d3041d3ae55265667b612f0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:49.584869 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a58cfa61d3041d3ae55265667b612f0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:49.585691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a58cfa61d3041d3ae55265667b612f0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:49.593548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a58cfa61d3041d3ae55265667b612f0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:49.607011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a58cfa61d3041d3ae55265667b612f0'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:49.616376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1079/2161] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:49 2022] DELETE /identity/v3/users/8a58cfa61d3041d3ae55265667b612f0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:49.621273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.621753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.621933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.640909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:49.642130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.708048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1083/2162] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:49.717975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.722504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.780061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3559f6-8e3a-44c4-9766-8c6b29a24c26 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.783955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe384-d3c0-4985-9d4b-63a564c20a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.784564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe384-d3c0-4985-9d4b-63a564c20a20 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.784851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe384-d3c0-4985-9d4b-63a564c20a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.785126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe384-d3c0-4985-9d4b-63a564c20a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.794923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfe384-d3c0-4985-9d4b-63a564c20a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.796349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfe384-d3c0-4985-9d4b-63a564c20a20 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.806274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1080/2163] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:50.080956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.082402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.094901 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a315709-7c98-4d7b-9068-12364accc5c2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.099405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.100032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.100310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.100579 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] PATH_INFO: `/v3/projects/e43f7a864fad4d659dbb7c785355fb5c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.101697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43f7a864fad4d659dbb7c785355fb5c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.104613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.161719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43f7a864fad4d659dbb7c785355fb5c', 'request_id': 'req-a705e2a1-ba01-4678-a7b4-774833c76a90'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.168749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43f7a864fad4d659dbb7c785355fb5c', 'request_id': 'req-a705e2a1-ba01-4678-a7b4-774833c76a90'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.175520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43f7a864fad4d659dbb7c785355fb5c', 'request_id': 'req-a705e2a1-ba01-4678-a7b4-774833c76a90'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.186076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] The token cache is being invalidate because project e43f7a864fad4d659dbb7c785355fb5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:50.186367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.187005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a705e2a1-ba01-4678-a7b4-774833c76a90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.188474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1084/2164] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:50 2022] DELETE /identity/v3/projects/e43f7a864fad4d659dbb7c785355fb5c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:50.193305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.193655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.193942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.214017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:50.215169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.232376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:50.276746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1081/2165] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:50.284491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.284784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.285061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.304948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:50.306809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.320204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1085/2166] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:50.325381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.335301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39690d74-9b3b-4780-b3c9-d5249b933c08 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.340344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.340973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.341257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.341593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.342268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:50.342576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.344199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.360574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1082/2167] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:50.361945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.374577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff08320-f906-4a1e-941c-42eb51db0976 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.379405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.380018 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.380299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.381080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.381252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.383218 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.413961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1086/2168] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:50.418978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.429086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c110-64d6-48fc-b0fa-ed0c213276ba admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.486478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.487141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.487281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.487573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.488196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.490144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.512825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1083/2169] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:50.516029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.526477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c38b8-9ca4-406c-8910-8237cb9e742e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.530999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.531597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.531874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.532147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.532777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.534394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.544656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1087/2170] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:50.548461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.559026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73276d7-b7c7-423c-8992-42d16f84be8d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.563707 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb356b-11d6-4cf8-b505-0c5e9f11af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.564297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb356b-11d6-4cf8-b505-0c5e9f11af89 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.564582 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb356b-11d6-4cf8-b505-0c5e9f11af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.564885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb356b-11d6-4cf8-b505-0c5e9f11af89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.565564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb356b-11d6-4cf8-b505-0c5e9f11af89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.567159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb356b-11d6-4cf8-b505-0c5e9f11af89 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.578802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1084/2171] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:50.582925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.593162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb05c7-0f2f-4d6f-a0ea-00695de71076 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.597841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.598488 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.598771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.599046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] PATH_INFO: `/v3/projects/282e3ae36d604ae295e4a1d056bfa5b0/users/78a300e01d8c440d82508ef27fc50a70/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.614224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] RBAC: Authorizing `identity:create_grant(project_id=282e3ae36d604ae295e4a1d056bfa5b0, user_id=78a300e01d8c440d82508ef27fc50a70, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.616914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.629917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1088/2172] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:50 2022] PUT /identity/v3/projects/282e3ae36d604ae295e4a1d056bfa5b0/users/78a300e01d8c440d82508ef27fc50a70/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:50.645277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.645594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.645873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.674091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] MFA Rules not processed for user `78a300e01d8c440d82508ef27fc50a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:50.675311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.744098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1085/2173] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:50.745366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.771340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:50.816816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcd77-8975-446c-88b5-85562cff367e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.822109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.822705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.822986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.823261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.823896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.825835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.858241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1089/2174] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:50.859446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.871712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9d1c-8dae-44b7-a966-14cf030e9123 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.890919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:50.943528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.944152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.944432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.944699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.945327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.947287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.972234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1086/2175] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:50.973216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.986273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0f506-59bb-4105-b772-d4f36cc300c6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.992217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.992658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.993134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.993531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.994256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.996165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.008612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1090/2176] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:51.012820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.023001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37826446-9f27-4f14-b979-5585082e6f22 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.029411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69449e33-0e94-4911-bc08-01d2500be276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.030114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69449e33-0e94-4911-bc08-01d2500be276 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.030392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69449e33-0e94-4911-bc08-01d2500be276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.030692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69449e33-0e94-4911-bc08-01d2500be276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.031333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69449e33-0e94-4911-bc08-01d2500be276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.032971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69449e33-0e94-4911-bc08-01d2500be276 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.043790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1087/2177] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:51.047911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.057911 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c0684-3bc5-4b28-b0f0-855e2d651094 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.062525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.063168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.063450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.063726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] PATH_INFO: `/v3/projects/13e8842e6e7645cfb79d4aff8f7b8135/users/3d4138a4796b4ba785f0bb53a9d4d570/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.078750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e8842e6e7645cfb79d4aff8f7b8135, user_id=3d4138a4796b4ba785f0bb53a9d4d570, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.081400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.095035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1091/2178] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:51 2022] PUT /identity/v3/projects/13e8842e6e7645cfb79d4aff8f7b8135/users/3d4138a4796b4ba785f0bb53a9d4d570/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:51.106897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.107186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.107554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.135712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] MFA Rules not processed for user `3d4138a4796b4ba785f0bb53a9d4d570`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:51.136928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:51.200728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1088/2179] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:51.205330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.266826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5abbf-2552-4de1-ac3b-9a4f624d7998 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.271354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.271961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.272244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.272520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.273424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.276101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.302262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1092/2180] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:51 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:51.306587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.317103 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd3638-1f31-4081-9403-a04cfa6cc4fb None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.373299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.373937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.374223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.374503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.375136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.377068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.400143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1089/2181] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:51 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:51.404413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.414755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eac8bd-ac42-4d0b-9641-d09ae26c8cc7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.419755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.420350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.420632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.420906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.421572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.423169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.433819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1093/2182] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:51.437676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.448217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f5ae6-adf9-401e-a282-b822e010b227 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.452765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.453414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.453727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.454000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] PATH_INFO: `/v3/projects/0a2415d4b5b74db2a9c4fd335f8adea7/users/6ce401d2ca3048eea32e381bef8ab902/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.466771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:51.470294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2415d4b5b74db2a9c4fd335f8adea7, user_id=6ce401d2ca3048eea32e381bef8ab902, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.473198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.489960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1090/2183] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:51 2022] PUT /identity/v3/projects/0a2415d4b5b74db2a9c4fd335f8adea7/users/6ce401d2ca3048eea32e381bef8ab902/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:51.494529 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.504228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c230a-0c76-415b-a68a-7af17db5653d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.548941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:51.566040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9617e-a699-4d57-93a1-7c1f351cf3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.566188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9617e-a699-4d57-93a1-7c1f351cf3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.566466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9617e-a699-4d57-93a1-7c1f351cf3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.566753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9617e-a699-4d57-93a1-7c1f351cf3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.567373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9617e-a699-4d57-93a1-7c1f351cf3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.569051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9617e-a699-4d57-93a1-7c1f351cf3f0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.579543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1094/2184] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:51.583621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.594299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8998de3-280f-4902-b248-88308228f0d6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.598905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.599559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.599833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.600102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] PATH_INFO: `/v3/projects/0a2415d4b5b74db2a9c4fd335f8adea7/users/6ce401d2ca3048eea32e381bef8ab902/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.602007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2415d4b5b74db2a9c4fd335f8adea7, user_id=6ce401d2ca3048eea32e381bef8ab902, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.605311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.616953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1091/2185] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:51 2022] PUT /identity/v3/projects/0a2415d4b5b74db2a9c4fd335f8adea7/users/6ce401d2ca3048eea32e381bef8ab902/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:51.626762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.627043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.627312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.656389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] MFA Rules not processed for user `6ce401d2ca3048eea32e381bef8ab902`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:51.657611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:51.725783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1095/2186] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:51.737972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.739797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:51.797876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd13c2-9cdd-417c-8048-f2d71ce3df47 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.801655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.802206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.802488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.802764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.858487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.859909 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.868858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1092/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:55.478235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:55.488349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df581a6-c891-465d-8aee-90637049aba1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:55.491976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:55.492588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.492867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.493142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.552290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:55.553781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.563062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1096/2188] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:56.116616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:56.125629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984ae6-18d7-495d-b6d4-fcae158d98c2 service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:56.129147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc09c60-4980-4e8f-9e51-a748da24c1a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:56.129778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09c60-4980-4e8f-9e51-a748da24c1a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:56.130057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09c60-4980-4e8f-9e51-a748da24c1a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:56.130330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09c60-4980-4e8f-9e51-a748da24c1a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:56.140120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc09c60-4980-4e8f-9e51-a748da24c1a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:56.141580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc09c60-4980-4e8f-9e51-a748da24c1a1 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:56.151930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1093/2189] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:58.950758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:58.952506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.042328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865f6dd-c096-4618-8b4f-cc69dd44d4c7 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.049725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.050377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.050651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.050927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] PATH_INFO: `/v3/users/87013d98ad33486098f785b70fd4e990` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.052071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87013d98ad33486098f785b70fd4e990)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.054053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.059520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:59.108611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87013d98ad33486098f785b70fd4e990'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.117863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87013d98ad33486098f785b70fd4e990'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.118546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87013d98ad33486098f785b70fd4e990'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.120109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87013d98ad33486098f785b70fd4e990'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.128574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87013d98ad33486098f785b70fd4e990'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.151344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1097/2190] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:58 2022] DELETE /identity/v3/users/87013d98ad33486098f785b70fd4e990 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:59.157178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.157624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.157773 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.179650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:59.180784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.247363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1094/2191] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:59.257733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.259279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.321295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834b9fc-3753-47db-b62f-26ff6cba3736 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.325414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.326044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.326319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.326589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.335855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.337500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.347880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1098/2192] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:59.806620 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.808105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.822057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad80603-be75-4353-ab71-b58b62beabf9 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.827472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.827687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.827958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.828236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] PATH_INFO: `/v3/projects/2a22890362174d059ac21917991ef68b` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.829375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a22890362174d059ac21917991ef68b)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.831337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.850609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:59.888789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a22890362174d059ac21917991ef68b', 'request_id': 'req-bd405e00-f024-4337-84a1-0ec9a1aec667'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.895482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a22890362174d059ac21917991ef68b', 'request_id': 'req-bd405e00-f024-4337-84a1-0ec9a1aec667'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.904609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a22890362174d059ac21917991ef68b', 'request_id': 'req-bd405e00-f024-4337-84a1-0ec9a1aec667'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.915575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] The token cache is being invalidate because project 2a22890362174d059ac21917991ef68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:59.915870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.916516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:59.918294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1095/2193] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:59 2022] DELETE /identity/v3/projects/2a22890362174d059ac21917991ef68b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:59.922192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.931921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d09a43-8099-4957-b550-786df9698dc0 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.988782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.989399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.989710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.989987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] PATH_INFO: `/v3/users/b176841e89b24932830fbcd69135d497` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.991123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b176841e89b24932830fbcd69135d497)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.993115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.027613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:00.051873 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b176841e89b24932830fbcd69135d497'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.061182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b176841e89b24932830fbcd69135d497'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.061910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b176841e89b24932830fbcd69135d497'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.062668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b176841e89b24932830fbcd69135d497'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.071561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e7e39cb0-3de9-483d-9383-6e4e67db5b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b176841e89b24932830fbcd69135d497'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.093708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1099/2194] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:59 2022] DELETE /identity/v3/users/b176841e89b24932830fbcd69135d497 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:00.096572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.096572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.110206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd405e00-f024-4337-84a1-0ec9a1aec667 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.170731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.171513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.171798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.172078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] PATH_INFO: `/v3/users/78a300e01d8c440d82508ef27fc50a70` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.173256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a300e01d8c440d82508ef27fc50a70)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.175417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.222537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a300e01d8c440d82508ef27fc50a70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.232358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a300e01d8c440d82508ef27fc50a70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.233208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a300e01d8c440d82508ef27fc50a70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.240522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a300e01d8c440d82508ef27fc50a70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.260126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a300e01d8c440d82508ef27fc50a70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.266300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1096/2195] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:00 2022] DELETE /identity/v3/users/78a300e01d8c440d82508ef27fc50a70 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:00.270865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.271105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.271411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.291202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:00.292435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.357050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1100/2196] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:00.372440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.373965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.390417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.391889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.404839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0442c4-cf12-46a1-bcf0-55c566ab2475 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.414452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.415071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.415360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.415689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] PATH_INFO: `/v3/projects/6defb039ba3548949c94fad3ee678ad1` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.416945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6defb039ba3548949c94fad3ee678ad1)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.419712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.435493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:00.473979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6defb039ba3548949c94fad3ee678ad1', 'request_id': 'req-58c6ba92-2bd0-4f47-832b-2259f2d8b758'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.475715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8efda1-3a16-4589-9824-9353609df930 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.483180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6defb039ba3548949c94fad3ee678ad1', 'request_id': 'req-58c6ba92-2bd0-4f47-832b-2259f2d8b758'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.492750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6defb039ba3548949c94fad3ee678ad1', 'request_id': 'req-58c6ba92-2bd0-4f47-832b-2259f2d8b758'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.505039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] The token cache is being invalidate because project 6defb039ba3548949c94fad3ee678ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:00.505328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.506034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58c6ba92-2bd0-4f47-832b-2259f2d8b758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.508100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1101/2197] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:00 2022] DELETE /identity/v3/projects/6defb039ba3548949c94fad3ee678ad1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:00.513328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0f5dc-e324-4cec-b829-bb5435fcaecc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.513452 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0f5dc-e324-4cec-b829-bb5435fcaecc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.513762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0f5dc-e324-4cec-b829-bb5435fcaecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.534726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2cf0f5dc-e324-4cec-b829-bb5435fcaecc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:00.535938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2cf0f5dc-e324-4cec-b829-bb5435fcaecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.543821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.544441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.544727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.545014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.546790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.561067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:00.601087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1102/2198] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:00.608709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.609003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.609288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.615275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.616718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.626792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1097/2199] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:02:00.630282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:00.631495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.641697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1103/2200] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:00.646008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.656584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c512-373c-4f7d-8eba-5413d484a958 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.661441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.662244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.662591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.662936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.663674 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:00.664160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.665913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.679813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1098/2201] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:00.685745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.695845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b9adf-34fa-4168-8310-6b29607e250e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.703085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.703739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.704051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.704689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.705186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.707139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.739459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1104/2202] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:00 2022] POST /identity/v3/projects => generated 407 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:00.740883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.751888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538a2d8-6a72-4c77-bb5a-2a1a6812c42e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.812720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.813254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.813569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.813851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.814483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.816490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.841489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1099/2203] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:00 2022] POST /identity/v3/users => generated 304 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:00.842768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.852900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c7dea-6422-412e-b01a-4b023c1172e7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.858751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.859364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.859646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.859922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.860580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.862201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.873422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1105/2204] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:00.877447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.885272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:00.891180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639e542-c887-46ec-a249-6b629ebc828f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.901084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.901084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.901999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.901999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.903350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.903463 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.905140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.905255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.922719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1100/2205] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:00.922933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0486-36aa-4b61-b069-bcaedf33179e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.924041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.927744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.928314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.928590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.928866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] PATH_INFO: `/v3/projects/282e3ae36d604ae295e4a1d056bfa5b0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.930224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282e3ae36d604ae295e4a1d056bfa5b0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.932131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.938821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b645-2002-4ad0-9bac-c20278c6c793 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.945204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b111009-a5ec-492f-a6b3-c8da0d4d7a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.946100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111009-a5ec-492f-a6b3-c8da0d4d7a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.946463 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111009-a5ec-492f-a6b3-c8da0d4d7a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.946821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111009-a5ec-492f-a6b3-c8da0d4d7a7c admin admin] PATH_INFO: `/v3/projects/b5ca81d72037461883761e4f9fc7221d/users/e8c5a8f47cbd4cf6b0fd5b2e7483481e/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.969236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b111009-a5ec-492f-a6b3-c8da0d4d7a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ca81d72037461883761e4f9fc7221d, user_id=e8c5a8f47cbd4cf6b0fd5b2e7483481e, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.972263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b111009-a5ec-492f-a6b3-c8da0d4d7a7c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.987290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1101/2206] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:00 2022] PUT /identity/v3/projects/b5ca81d72037461883761e4f9fc7221d/users/e8c5a8f47cbd4cf6b0fd5b2e7483481e/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:00.995818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282e3ae36d604ae295e4a1d056bfa5b0', 'request_id': 'req-642e2d01-7845-4480-8578-32792d064cc9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:00.999118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.999478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.999852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.005072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282e3ae36d604ae295e4a1d056bfa5b0', 'request_id': 'req-642e2d01-7845-4480-8578-32792d064cc9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.014029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282e3ae36d604ae295e4a1d056bfa5b0', 'request_id': 'req-642e2d01-7845-4480-8578-32792d064cc9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.025377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] The token cache is being invalidate because project 282e3ae36d604ae295e4a1d056bfa5b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:01.025700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.027074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.028851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1106/2207] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:00 2022] DELETE /identity/v3/projects/282e3ae36d604ae295e4a1d056bfa5b0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:01.032119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] MFA Rules not processed for user `e8c5a8f47cbd4cf6b0fd5b2e7483481e`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:01.032729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.033801 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.034103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.110775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2d01-7845-4480-8578-32792d064cc9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.114651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1102/2208] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:01.119262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.120624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.134377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:01.177851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.178518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.178792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.179064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] PATH_INFO: `/v3/users/3d4138a4796b4ba785f0bb53a9d4d570` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.180204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4138a4796b4ba785f0bb53a9d4d570)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.182184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.192248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e13fa1-b0e6-47da-9fc8-6688b5f4f418 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.196983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.197630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.197912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.198239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.198820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.201617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.229322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1103/2209] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:01 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:01.232411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4138a4796b4ba785f0bb53a9d4d570'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.233873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.240912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4138a4796b4ba785f0bb53a9d4d570'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.241673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4138a4796b4ba785f0bb53a9d4d570'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.242293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4138a4796b4ba785f0bb53a9d4d570'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.250101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4138a4796b4ba785f0bb53a9d4d570'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.270979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1107/2210] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:01 2022] DELETE /identity/v3/users/3d4138a4796b4ba785f0bb53a9d4d570 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:01.281948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:01.300512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadab0c-985b-47da-9c43-d7e4bf2c76e3 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.361049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.361622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.361906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.362184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.362816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.364744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.388192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1104/2211] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:01 2022] POST /identity/v3/users => generated 310 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:01.392230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.393753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.406915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc2b77-1ecc-46f3-9e3a-e117cb39597d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.411700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.412411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.412777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.413060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.413735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.415328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.426450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1108/2212] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:01.430654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.441437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea376b35-7812-469a-ba85-2de6ecee8ca5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.447187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.447851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.448136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.448411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] PATH_INFO: `/v3/projects/57f2605d69ad419fb7430e647ba44601/users/8d6fa5ffaf304bee9892e34cb5cb4874/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.465121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f2605d69ad419fb7430e647ba44601, user_id=8d6fa5ffaf304bee9892e34cb5cb4874, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.468134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.490126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1105/2213] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:01 2022] PUT /identity/v3/projects/57f2605d69ad419fb7430e647ba44601/users/8d6fa5ffaf304bee9892e34cb5cb4874/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:01.494271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.505561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6d410-df9f-49f2-99fb-23c80cd82089 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.569946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa1799-295d-4e16-be5c-4595beb7aadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.570673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa1799-295d-4e16-be5c-4595beb7aadc admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.571021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa1799-295d-4e16-be5c-4595beb7aadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.571369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa1799-295d-4e16-be5c-4595beb7aadc admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.572085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa1799-295d-4e16-be5c-4595beb7aadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.573875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa1799-295d-4e16-be5c-4595beb7aadc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.585827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1109/2214] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:01.590714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.601414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23696224-3f3d-4160-9211-40eddad177fd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.606235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.606910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.607188 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.607459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] PATH_INFO: `/v3/projects/57f2605d69ad419fb7430e647ba44601/users/8d6fa5ffaf304bee9892e34cb5cb4874/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.609381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f2605d69ad419fb7430e647ba44601, user_id=8d6fa5ffaf304bee9892e34cb5cb4874, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.612184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.623571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1106/2215] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:01 2022] PUT /identity/v3/projects/57f2605d69ad419fb7430e647ba44601/users/8d6fa5ffaf304bee9892e34cb5cb4874/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:01.633857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.634184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.634420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.667699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] MFA Rules not processed for user `8d6fa5ffaf304bee9892e34cb5cb4874`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:01.669820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.683622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:01.753543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1110/2216] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:01.765189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.767158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.785952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.787503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.837702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86efc0-25c6-4454-934e-d581acdaa255 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.842149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.842766 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.843050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.843323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.852538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.854022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.855793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81570bd4-77fd-4cab-bff5-8ec8f08b68ab None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.860496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.861136 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.861411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.861713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] PATH_INFO: `/v3/projects/13e8842e6e7645cfb79d4aff8f7b8135` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.862837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e8842e6e7645cfb79d4aff8f7b8135)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.863790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1107/2217] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:01.864685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.917592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e8842e6e7645cfb79d4aff8f7b8135', 'request_id': 'req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.924394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e8842e6e7645cfb79d4aff8f7b8135', 'request_id': 'req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.932434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e8842e6e7645cfb79d4aff8f7b8135', 'request_id': 'req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.945264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] The token cache is being invalidate because project 13e8842e6e7645cfb79d4aff8f7b8135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:01.945596 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.946254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.947759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1111/2218] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:01 2022] DELETE /identity/v3/projects/13e8842e6e7645cfb79d4aff8f7b8135 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:01.951808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.962392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538cc6-83cd-4a15-a296-103e38eea87b service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:02.011178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:02.028263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:02.028433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:02.029011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.029406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.029732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] PATH_INFO: `/v3/users/6ce401d2ca3048eea32e381bef8ab902` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.030163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.030974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce401d2ca3048eea32e381bef8ab902)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:02.032916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:02.079680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:02.088996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce401d2ca3048eea32e381bef8ab902'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.098298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e784d-1d45-44d8-bb0a-6eb1ef508bec admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:02.098476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce401d2ca3048eea32e381bef8ab902'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.098991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce401d2ca3048eea32e381bef8ab902'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.102630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:02.103243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.103527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.103799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.106419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.108982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce401d2ca3048eea32e381bef8ab902'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.123405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce401d2ca3048eea32e381bef8ab902'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.130125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1108/2219] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:01 2022] DELETE /identity/v3/users/6ce401d2ca3048eea32e381bef8ab902 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:02.181747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:02.183187 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438c71e-47f5-4179-920e-1f73d8c55295 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:02.203859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1112/2220] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:02 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:02:02.611857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:02.614013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.724168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678e6d-7ba1-48c4-83de-1c6caefb5450 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:02.730940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:02.731608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.731904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.732183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] PATH_INFO: `/v3/projects/0a2415d4b5b74db2a9c4fd335f8adea7` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.734654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2415d4b5b74db2a9c4fd335f8adea7)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:02.742880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:02.793729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2415d4b5b74db2a9c4fd335f8adea7', 'request_id': 'req-792d1c74-7458-417c-a91d-f536a96b522f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.801114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2415d4b5b74db2a9c4fd335f8adea7', 'request_id': 'req-792d1c74-7458-417c-a91d-f536a96b522f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.810607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2415d4b5b74db2a9c4fd335f8adea7', 'request_id': 'req-792d1c74-7458-417c-a91d-f536a96b522f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.816606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:02.829364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] The token cache is being invalidate because project 0a2415d4b5b74db2a9c4fd335f8adea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:02.829962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.831076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-792d1c74-7458-417c-a91d-f536a96b522f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.834468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1109/2221] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:02 2022] DELETE /identity/v3/projects/0a2415d4b5b74db2a9c4fd335f8adea7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:02.840293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.840643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.840984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.865304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:02.866573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.951016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1113/2222] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:02.960212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.960538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.960816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.983609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:02.984845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.993788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1110/2223] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:02.999119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.009206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08d94a-3e3f-493d-919d-ea5a30316a6b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.014601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.015211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.015493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.015770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.016418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:03.016725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.018364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.032063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1114/2224] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:03.036498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.047246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d25ef-9ad5-4fc0-bf45-e35d6f0283ed None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.052817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.053419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.053727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.053999 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.054642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.056556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.087578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1111/2225] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:03.092646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.103926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8761-5af0-4c59-977b-92d4f3d117be admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.175868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.176390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.176666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.176935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.177595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.179501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.202027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1115/2226] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:03.206366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.217855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519524-5385-4c68-98fa-b785ad37076c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.222417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.223017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.223302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.223580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.224221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.225938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.236036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1112/2227] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:03.239957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.250025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21365158-b58b-4649-a5d9-5ea121824fa8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.254543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a281cb-519b-4eea-ac87-d35466949498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.255130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a281cb-519b-4eea-ac87-d35466949498 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.255407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a281cb-519b-4eea-ac87-d35466949498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.255676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a281cb-519b-4eea-ac87-d35466949498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.256320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a281cb-519b-4eea-ac87-d35466949498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.257922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a281cb-519b-4eea-ac87-d35466949498 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.263532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:03.270208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1116/2228] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:03.274353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.284980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b21bdd-cb39-4356-a18c-7a8c0d5defb5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.289533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.290176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.290454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.290727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] PATH_INFO: `/v3/projects/d04f0d41be0c4be6a474a67c226065f8/users/dcce3ee14870450abe122ea7219de4c3/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.305956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04f0d41be0c4be6a474a67c226065f8, user_id=dcce3ee14870450abe122ea7219de4c3, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.308650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.321191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1113/2229] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:03 2022] PUT /identity/v3/projects/d04f0d41be0c4be6a474a67c226065f8/users/dcce3ee14870450abe122ea7219de4c3/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:03.388299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b96a6f-b02b-425b-8b4d-d32df29998ce None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.388588 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b96a6f-b02b-425b-8b4d-d32df29998ce None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.388871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b96a6f-b02b-425b-8b4d-d32df29998ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.417362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-67b96a6f-b02b-425b-8b4d-d32df29998ce None None] MFA Rules not processed for user `dcce3ee14870450abe122ea7219de4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:03.418609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-67b96a6f-b02b-425b-8b4d-d32df29998ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:03.488084 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1117/2230] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:03.501981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.503811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:03.563779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b30ae-d88a-4f91-bce2-ae4e4b665da9 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.567799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.568402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.568690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.568963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.578285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.579749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.588727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1114/2231] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:07.147515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb879a7-9157-4795-b184-5808de04b366 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.148298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb879a7-9157-4795-b184-5808de04b366 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.148298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb879a7-9157-4795-b184-5808de04b366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.169852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ffb879a7-9157-4795-b184-5808de04b366 None None] MFA Rules not processed for user `dcce3ee14870450abe122ea7219de4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:07.170990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ffb879a7-9157-4795-b184-5808de04b366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:07.178067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1118/2232] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:11.878305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:11.881658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:11.972054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8aa0-c311-4be0-8629-e544229fba6f service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:11.979403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:11.981447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:11.981752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:11.982253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] PATH_INFO: `/v3/users/e8c5a8f47cbd4cf6b0fd5b2e7483481e` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:11.983359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c5a8f47cbd4cf6b0fd5b2e7483481e)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:11.986649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:12.034919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:12.045945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c5a8f47cbd4cf6b0fd5b2e7483481e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:12.055845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c5a8f47cbd4cf6b0fd5b2e7483481e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:12.056047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c5a8f47cbd4cf6b0fd5b2e7483481e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:12.065516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c5a8f47cbd4cf6b0fd5b2e7483481e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:12.085641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c5a8f47cbd4cf6b0fd5b2e7483481e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:12.104363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1115/2233] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:11 2022] DELETE /identity/v3/users/e8c5a8f47cbd4cf6b0fd5b2e7483481e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:12.110427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:12.111436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:12.111959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:12.137985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:12.139550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:12.219159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1119/2234] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:12.241408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:12.242921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:12.355540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a9cc6-e0c5-4034-a610-41037310cdf8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:12.360549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:12.361290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:12.361706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:12.362088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:12.372562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:12.374158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:12.387316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1116/2235] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:13.126937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:13.128389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:13.154645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da089-5004-4f98-8b38-669e92cc83fd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:13.160522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:13.161693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:13.162017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:13.162270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] PATH_INFO: `/v3/projects/b5ca81d72037461883761e4f9fc7221d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:13.163550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ca81d72037461883761e4f9fc7221d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:13.165741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:13.233630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ca81d72037461883761e4f9fc7221d', 'request_id': 'req-6c8b2073-2b53-48d5-b994-e510307fa8fc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.240531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ca81d72037461883761e4f9fc7221d', 'request_id': 'req-6c8b2073-2b53-48d5-b994-e510307fa8fc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.249403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ca81d72037461883761e4f9fc7221d', 'request_id': 'req-6c8b2073-2b53-48d5-b994-e510307fa8fc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.262276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] The token cache is being invalidate because project b5ca81d72037461883761e4f9fc7221d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:13.262796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.263626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c8b2073-2b53-48d5-b994-e510307fa8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.265344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1120/2236] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:13 2022] DELETE /identity/v3/projects/b5ca81d72037461883761e4f9fc7221d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:13.269487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:13.281630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96fbab-0dc5-41b9-bc8b-4ef3016fa57b service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:13.314163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:13.314457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:13.314750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:13.330710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:13.336653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] MFA Rules not processed for user `47f9883392364d3fb5d0b921d6f451d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:13.338532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:13.385380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:13.386090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:13.386409 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:13.386692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] PATH_INFO: `/v3/users/8d6fa5ffaf304bee9892e34cb5cb4874` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:13.394790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6fa5ffaf304bee9892e34cb5cb4874)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:13.397438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:13.422029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1121/2237] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:13.456501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6fa5ffaf304bee9892e34cb5cb4874'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.466844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6fa5ffaf304bee9892e34cb5cb4874'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.467733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6fa5ffaf304bee9892e34cb5cb4874'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.472954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:13.476072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6fa5ffaf304bee9892e34cb5cb4874'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.494015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-79c104bf-f72e-4d5e-8e60-6ec1e2b0cc6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6fa5ffaf304bee9892e34cb5cb4874'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:13.501090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1117/2238] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:13 2022] DELETE /identity/v3/users/8d6fa5ffaf304bee9892e34cb5cb4874 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:14.132183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.133778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:14.164608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3f984-86e6-4744-aa2e-1c5821b876f0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.234330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.235085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.235422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.235780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] PATH_INFO: `/v3/projects/57f2605d69ad419fb7430e647ba44601` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.236953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f2605d69ad419fb7430e647ba44601)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.238963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.289945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f2605d69ad419fb7430e647ba44601', 'request_id': 'req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.296668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f2605d69ad419fb7430e647ba44601', 'request_id': 'req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.305524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f2605d69ad419fb7430e647ba44601', 'request_id': 'req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.317807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] The token cache is being invalidate because project 57f2605d69ad419fb7430e647ba44601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:14.318110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.318776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-789cccaa-67b3-4bf9-93af-e8ee1ecc7738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.320322 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1122/2239] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:14 2022] DELETE /identity/v3/projects/57f2605d69ad419fb7430e647ba44601 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:14.325684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.326181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.326321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.347295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:14.348527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:14.423541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1118/2240] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:14.431949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.432234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.432510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.455451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:14.456721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:14.481668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1123/2241] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:14.485946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.496637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffbe05-bfc0-4341-984c-4ab0acfbbd06 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.501292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.501972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.502255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.502531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.503185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:14.503493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.505129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.517953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1119/2242] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:14.522249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.532883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98ff42-ad09-43c0-a7f8-adbe7e73077d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.538542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.539159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.539442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.539717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.540427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.543998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.582813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1124/2243] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:14 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:14.583820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.596202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2df8-fcfb-4bcd-85fb-f7eea811a246 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.664769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.665393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.665704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.666058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.666706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.668679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.705550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1120/2244] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:14 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:14.708434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.719954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a9111-5fc5-44c7-8d62-944785872715 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.724555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.725159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.725435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.725736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.726370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.728015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.739367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1125/2245] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:14.750106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.764978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530dec8-ebc4-402b-9f5a-41135adc0735 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.770289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb2ec7-9033-44d9-9e5b-0c7deaf39781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.770943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb2ec7-9033-44d9-9e5b-0c7deaf39781 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.771233 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb2ec7-9033-44d9-9e5b-0c7deaf39781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.771533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb2ec7-9033-44d9-9e5b-0c7deaf39781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.772201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb2ec7-9033-44d9-9e5b-0c7deaf39781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.774163 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb2ec7-9033-44d9-9e5b-0c7deaf39781 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.786836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1121/2246] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:14.791047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.797554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:14.804157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e08c5-6055-4e64-81f3-e328ad169d59 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.813153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.814114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.815300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.815864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] PATH_INFO: `/v3/projects/e03c5b951eff4807973949f288fef416/users/b522195f840c4e42acc3cdf53ee72ba4/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.833258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03c5b951eff4807973949f288fef416, user_id=b522195f840c4e42acc3cdf53ee72ba4, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.835975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.863067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1126/2247] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:14 2022] PUT /identity/v3/projects/e03c5b951eff4807973949f288fef416/users/b522195f840c4e42acc3cdf53ee72ba4/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:14.863812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.864159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.864504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.908587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] MFA Rules not processed for user `b522195f840c4e42acc3cdf53ee72ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:14.909928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:14.949169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:14.994009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1122/2248] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:14.998960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.078043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f0082-5e70-4b72-9fb0-b602d823ed4b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.083795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.084477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.084758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.085030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.085800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.088113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.141311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1127/2249] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:14 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:15.143149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.157936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9cdb5-6aba-4ab2-b385-92e739830071 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.240736 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.241547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.241833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.242108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.242770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.244992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.274249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1123/2250] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:15 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:15.278113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.289514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46017299-3d73-4a38-95bc-6fec6588d02c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.296238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.296856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.297297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.297760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.298413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.300468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.318375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1128/2251] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:15.320191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.331703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199a044-2c2f-4584-90d7-205fa8f5c4b2 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.338697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.340393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.340821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.341195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] PATH_INFO: `/v3/projects/d33d50f3d84f4663b38e5e5897e8001d/users/39736c07f37643e48d512051fef04727/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.363381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33d50f3d84f4663b38e5e5897e8001d, user_id=39736c07f37643e48d512051fef04727, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.366555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.381452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1124/2252] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:15 2022] PUT /identity/v3/projects/d33d50f3d84f4663b38e5e5897e8001d/users/39736c07f37643e48d512051fef04727/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:15.385356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.396517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b140ac0-d792-48c5-be39-9ead3ccf8b89 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.457053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104daf06-28c2-48e9-9480-1b0291a5064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.457694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104daf06-28c2-48e9-9480-1b0291a5064d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.457977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104daf06-28c2-48e9-9480-1b0291a5064d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.458252 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104daf06-28c2-48e9-9480-1b0291a5064d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.458891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104daf06-28c2-48e9-9480-1b0291a5064d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.460471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104daf06-28c2-48e9-9480-1b0291a5064d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.471293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1129/2253] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:02:15.475179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.499906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19340f-d6bb-49cd-a31e-a90c870e6b06 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.509035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.510128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.510557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.510979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] PATH_INFO: `/v3/projects/d33d50f3d84f4663b38e5e5897e8001d/users/39736c07f37643e48d512051fef04727/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.514974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33d50f3d84f4663b38e5e5897e8001d, user_id=39736c07f37643e48d512051fef04727, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.518449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.531360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1125/2254] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:15 2022] PUT /identity/v3/projects/d33d50f3d84f4663b38e5e5897e8001d/users/39736c07f37643e48d512051fef04727/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:15.547713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.547713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.547713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.572212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] MFA Rules not processed for user `39736c07f37643e48d512051fef04727`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:15.573430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:15.612489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:15.643470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1130/2255] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:15.655837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:15.657537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:15.717805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966701db-e40d-4815-814b-d0ab7bf3bf26 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:15.721448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2aab1-69b3-46dc-b1eb-ee608be12e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:15.722081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2aab1-69b3-46dc-b1eb-ee608be12e56 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:15.722365 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2aab1-69b3-46dc-b1eb-ee608be12e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:15.722649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2aab1-69b3-46dc-b1eb-ee608be12e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:15.768196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:15.788650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2aab1-69b3-46dc-b1eb-ee608be12e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:15.790140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2aab1-69b3-46dc-b1eb-ee608be12e56 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:15.800093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1126/2256] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:18.390980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:18.467159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c1a81-718c-443c-960b-5f6538148c6c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:18.472028 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:18.473090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:18.473090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:18.473455 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] PATH_INFO: `/v3/users/b522195f840c4e42acc3cdf53ee72ba4` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:18.474627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b522195f840c4e42acc3cdf53ee72ba4)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:18.476564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:18.513167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:18.513745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:18.513930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:18.538912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:18.540497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:18.551839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b522195f840c4e42acc3cdf53ee72ba4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:18.564755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b522195f840c4e42acc3cdf53ee72ba4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:18.565432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b522195f840c4e42acc3cdf53ee72ba4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:18.566149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b522195f840c4e42acc3cdf53ee72ba4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:18.577522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fb91dcab-3f9b-426d-aeff-32faff4b888c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b522195f840c4e42acc3cdf53ee72ba4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:18.607226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1131/2257] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:18 2022] DELETE /identity/v3/users/b522195f840c4e42acc3cdf53ee72ba4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:18.611751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:18.612255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:18.612653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:18.638915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:18.640262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:18.644209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1127/2258] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:18.708034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:18.712325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:18.714958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:18.746392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1132/2259] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:18.753114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:18.754918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:18.851788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782a30e-fd07-4ada-91e9-b522418697cc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:18.857562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:18.858398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:18.858899 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:18.859593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:18.877999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:18.880027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:18.907881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1133/2260] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:18.912795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824c6ad-087f-4acd-9ba7-c0b8be40ffb6 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:19.047702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:19.048559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:19.048910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:19.049404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:19.109571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:19.165780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:19.167947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:19.178563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1128/2261] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:02:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:02:19.541645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:19.543535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:19.568138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dc3db-7e6b-4b02-b81e-68b49e93b6c7 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:19.574306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:19.575295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:19.575660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:19.576003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] PATH_INFO: `/v3/projects/e03c5b951eff4807973949f288fef416` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:19.577213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03c5b951eff4807973949f288fef416)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:19.579329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:19.641806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03c5b951eff4807973949f288fef416', 'request_id': 'req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.653545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03c5b951eff4807973949f288fef416', 'request_id': 'req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.664140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03c5b951eff4807973949f288fef416', 'request_id': 'req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.681977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] The token cache is being invalidate because project e03c5b951eff4807973949f288fef416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:19.682512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.683320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.689852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:19.690408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1134/2262] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:19 2022] DELETE /identity/v3/projects/e03c5b951eff4807973949f288fef416 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:19.704838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7fe49-d2ed-4478-b69e-1f9afa11bac1 service swift] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:19.773364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:19.774288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:19.774636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:19.775009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] PATH_INFO: `/v3/users/39736c07f37643e48d512051fef04727` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:19.776200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=39736c07f37643e48d512051fef04727)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:19.778859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:19.858058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39736c07f37643e48d512051fef04727'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.873874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39736c07f37643e48d512051fef04727'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.875210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39736c07f37643e48d512051fef04727'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.892635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39736c07f37643e48d512051fef04727'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.916464 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9bfc14f3-0c47-45a0-9d77-a07b20294dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39736c07f37643e48d512051fef04727'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:19.925982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1129/2263] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:19 2022] DELETE /identity/v3/users/39736c07f37643e48d512051fef04727 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:20.403679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:20.405186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:20.422483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634dbe5-6cd9-4eb8-a8c7-5eb28a91bfd7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:20.576886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:20.578262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:20.578706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:20.579235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] PATH_INFO: `/v3/projects/d33d50f3d84f4663b38e5e5897e8001d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:20.583262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33d50f3d84f4663b38e5e5897e8001d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:20.585690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:20.648794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:20.722723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33d50f3d84f4663b38e5e5897e8001d', 'request_id': 'req-5163f4b0-f8f5-4865-9316-654b7632931a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:20.737950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33d50f3d84f4663b38e5e5897e8001d', 'request_id': 'req-5163f4b0-f8f5-4865-9316-654b7632931a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:20.751528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33d50f3d84f4663b38e5e5897e8001d', 'request_id': 'req-5163f4b0-f8f5-4865-9316-654b7632931a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:20.766304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] The token cache is being invalidate because project d33d50f3d84f4663b38e5e5897e8001d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:20.766600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:20.767283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5163f4b0-f8f5-4865-9316-654b7632931a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:20.773062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1135/2264] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:20 2022] DELETE /identity/v3/projects/d33d50f3d84f4663b38e5e5897e8001d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:20.774282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:20.774728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:20.774816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:20.817200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:20.818472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:20.910905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:20.954985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1130/2265] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:20.965214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:20.965626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:20.965908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:20.989611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:20.991059 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:21.000217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1136/2266] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:21.004826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.016510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c85be1-2bd5-4b5e-9de0-3ba19dc6ae8c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.023111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.024164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.024556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.024932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.025927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:21.026356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.029388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.048802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1131/2267] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:21.050031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.062575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6e3a4-3b28-4484-be6d-355f4c5394d6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.081852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.082180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.082667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.083158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.084352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.090000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.138204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1137/2268] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:21 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:21.142340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.161716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb8585-769b-4b50-a3f7-6afc03502acd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.287483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.288170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.288484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.289003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.289670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.291596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.324818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1132/2269] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:21 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:21.326017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.338301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaa2b8-0503-482b-a7b0-20562a311960 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.343991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.344671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.350457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.350843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.351551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.353193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.367940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1138/2270] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:21.370994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.384578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad6964-e5d6-4c25-8dda-0c27166b5a50 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.393259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92f1f2-c7c2-4b1c-be8c-a0e5d06b3ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.394087 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f1f2-c7c2-4b1c-be8c-a0e5d06b3ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.394436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f1f2-c7c2-4b1c-be8c-a0e5d06b3ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.394776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f1f2-c7c2-4b1c-be8c-a0e5d06b3ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.397747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92f1f2-c7c2-4b1c-be8c-a0e5d06b3ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.400296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92f1f2-c7c2-4b1c-be8c-a0e5d06b3ee3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.420809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1133/2271] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:21.421445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.443398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff99a-998d-4f6f-84f0-03f273a6492d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.457782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.458703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.459174 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.459697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] PATH_INFO: `/v3/projects/12e813ffd18d40b9be703ff655385397/users/0e2ccb201b434349a8918e8094aa2950/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.507591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e813ffd18d40b9be703ff655385397, user_id=0e2ccb201b434349a8918e8094aa2950, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.510312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.564318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1139/2272] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:21 2022] PUT /identity/v3/projects/12e813ffd18d40b9be703ff655385397/users/0e2ccb201b434349a8918e8094aa2950/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:21.577285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a0200-368c-4a53-867e-256b3151c4d9 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.578190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a0200-368c-4a53-867e-256b3151c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.582918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a0200-368c-4a53-867e-256b3151c4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.633591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ba7a0200-368c-4a53-867e-256b3151c4d9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:21.633591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ba7a0200-368c-4a53-867e-256b3151c4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:21.720296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1134/2273] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:21.732188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.734298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:21.822996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd038e-aeca-4d60-864c-c974e2ff292e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.828758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.829853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.830205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.830549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.842414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.843954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.856065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1140/2274] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:22.861686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b45df-b669-42a4-942c-0873cf66701d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:22.862221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b45df-b669-42a4-942c-0873cf66701d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:22.862221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b45df-b669-42a4-942c-0873cf66701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:22.912653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-620b45df-b669-42a4-942c-0873cf66701d None None] MFA Rules not processed for user `0e2ccb201b434349a8918e8094aa2950`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:22.913366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-620b45df-b669-42a4-942c-0873cf66701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:22.978498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:23.011253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1135/2275] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:23.018817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:23.020460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:23.122570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:23.137587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf027c-8de1-4848-8b90-26eb30797c3c service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:23.150195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b904a2-b22a-4107-840f-b639d5ff51c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:23.150983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b904a2-b22a-4107-840f-b639d5ff51c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:23.155008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b904a2-b22a-4107-840f-b639d5ff51c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:23.155443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b904a2-b22a-4107-840f-b639d5ff51c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:23.180735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b904a2-b22a-4107-840f-b639d5ff51c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:23.182428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b904a2-b22a-4107-840f-b639d5ff51c5 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:23.201594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1141/2276] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:24.864910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313217af-329d-49bf-b283-1ca133e466d6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:24.867969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313217af-329d-49bf-b283-1ca133e466d6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:24.872015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313217af-329d-49bf-b283-1ca133e466d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:24.930661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-313217af-329d-49bf-b283-1ca133e466d6 None None] MFA Rules not processed for user `0e2ccb201b434349a8918e8094aa2950`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:24.932049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-313217af-329d-49bf-b283-1ca133e466d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:24.948606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1136/2277] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:31.473357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:31.476047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:31.477076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:31.517964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] MFA Rules not processed for user `16610ae2c3214396b78fb1caff01ab6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:31.519965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:31.623409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1142/2278] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:31.775311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a234-d807-4ba0-bd0a-3116ae4e398d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:31.776855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a234-d807-4ba0-bd0a-3116ae4e398d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:31.777936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a234-d807-4ba0-bd0a-3116ae4e398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:31.799158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ab93a234-d807-4ba0-bd0a-3116ae4e398d None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:31.800610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ab93a234-d807-4ba0-bd0a-3116ae4e398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:31.879140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1137/2279] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:31.892881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:31.894446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:31.991694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35405c8c-acf5-413e-8627-48514cc65bc0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:31.999334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:32.000036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:32.001315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:32.001315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:32.011272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:32.012958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:32.023533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1143/2280] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:02:32.757942 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5a621-2d31-4c1d-b20a-dbc11763096e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:32.758605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5a621-2d31-4c1d-b20a-dbc11763096e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:32.759165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5a621-2d31-4c1d-b20a-dbc11763096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:32.795704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-80c5a621-2d31-4c1d-b20a-dbc11763096e None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:32.798052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-80c5a621-2d31-4c1d-b20a-dbc11763096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:32.831639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1138/2281] 10.222.0.23 () {58 vars in 967 bytes} [Sat Sep 10 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:32.832605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:32.846016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb4bc1-a422-48f7-a251-c8674991bd66 service swift] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:32.851704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:32.852656 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:32.853143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:32.855302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:32.866327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:32.888037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:32.907260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1144/2282] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:02:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:02:33.435519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015676-6dc6-467e-ba48-752ffcd4f784 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:33.436109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015676-6dc6-467e-ba48-752ffcd4f784 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:33.436619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015676-6dc6-467e-ba48-752ffcd4f784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:33.466290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-84015676-6dc6-467e-ba48-752ffcd4f784 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:33.467871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-84015676-6dc6-467e-ba48-752ffcd4f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:33.480758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1139/2283] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:02:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:33.496959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:33.512171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf4073-c7cc-43d0-98c7-dddf1b628d7f service swift] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:33.521373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:33.522905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:33.523511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:33.524944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:33.537724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:33.540105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:33.556207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1145/2284] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:02:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:02:34.009789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.010358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.010446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.031435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:34.032643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:34.042473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1140/2285] 10.222.0.23 () {58 vars in 967 bytes} [Sat Sep 10 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:34.054560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:34.067716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890aecd-c7e9-4325-ae9a-ee4e1a167762 service swift] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:34.073891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d776a-0b80-42bf-b8cb-b79007905b9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:34.074644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d776a-0b80-42bf-b8cb-b79007905b9b service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.074936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d776a-0b80-42bf-b8cb-b79007905b9b service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.075212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d776a-0b80-42bf-b8cb-b79007905b9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.086181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d776a-0b80-42bf-b8cb-b79007905b9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:34.087818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d776a-0b80-42bf-b8cb-b79007905b9b service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:34.101663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1146/2286] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:02:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:02:36.328510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.330043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:36.345229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bb6f-ebab-413d-a823-3351c9fe16f0 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.349992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.350619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.350899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.351178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] PATH_INFO: `/v3/users/dcce3ee14870450abe122ea7219de4c3` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.353185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcce3ee14870450abe122ea7219de4c3)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.355504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.428976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcce3ee14870450abe122ea7219de4c3'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.446881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcce3ee14870450abe122ea7219de4c3'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.450739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcce3ee14870450abe122ea7219de4c3'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.457519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcce3ee14870450abe122ea7219de4c3'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.471082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcce3ee14870450abe122ea7219de4c3'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.478299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1141/2287] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:36 2022] DELETE /identity/v3/users/dcce3ee14870450abe122ea7219de4c3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:36.482792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.483628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.484162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.513814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:36.515043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:36.568180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:36.596323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1147/2288] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:36.606963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.608412 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:36.644888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:36.673175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2efd9-a0d9-4d6f-827c-c24c1b293272 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.677073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e033eb8-5dee-4faa-83c0-f10f81bc1fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.677844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e033eb8-5dee-4faa-83c0-f10f81bc1fea service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.678126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e033eb8-5dee-4faa-83c0-f10f81bc1fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.678398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e033eb8-5dee-4faa-83c0-f10f81bc1fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.687895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e033eb8-5dee-4faa-83c0-f10f81bc1fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.689353 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e033eb8-5dee-4faa-83c0-f10f81bc1fea service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.698986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1142/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:36.871703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.873328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:36.889154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab9e4-37ea-4231-beb6-f3814fa965b8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.894241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.894893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.895172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.895445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] PATH_INFO: `/v3/projects/d04f0d41be0c4be6a474a67c226065f8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.896685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04f0d41be0c4be6a474a67c226065f8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.899429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.981444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04f0d41be0c4be6a474a67c226065f8', 'request_id': 'req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.989485 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04f0d41be0c4be6a474a67c226065f8', 'request_id': 'req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:36.998864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04f0d41be0c4be6a474a67c226065f8', 'request_id': 'req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.013030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] The token cache is being invalidate because project d04f0d41be0c4be6a474a67c226065f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:37.013426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.014266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2f97a71-cea1-4b60-b853-39ecbdf60ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.015785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1148/2290] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:36 2022] DELETE /identity/v3/projects/d04f0d41be0c4be6a474a67c226065f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:37.556364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816ad0-3d8b-4ee2-b317-102e1b67c3be None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:37.557368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816ad0-3d8b-4ee2-b317-102e1b67c3be None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:37.557921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816ad0-3d8b-4ee2-b317-102e1b67c3be None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:37.560108 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1143/2291] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:02:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:37.567986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908b448-9e35-494e-8c64-f33d943a8839 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:37.568980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908b448-9e35-494e-8c64-f33d943a8839 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:37.569547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908b448-9e35-494e-8c64-f33d943a8839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:37.633436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0908b448-9e35-494e-8c64-f33d943a8839 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:37.638823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0908b448-9e35-494e-8c64-f33d943a8839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:37.808713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1149/2292] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:02:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:37.826604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816ad0-3d8b-4ee2-b317-102e1b67c3be None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:37.829287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d9816ad0-3d8b-4ee2-b317-102e1b67c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:37.938778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816ad0-3d8b-4ee2-b317-102e1b67c3be None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:37.942423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd92fc-8ed5-4646-8cb0-6ffb923869a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:37.943043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd92fc-8ed5-4646-8cb0-6ffb923869a2 service glance] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:37.943333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd92fc-8ed5-4646-8cb0-6ffb923869a2 service glance] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:37.943615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd92fc-8ed5-4646-8cb0-6ffb923869a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:37.955915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd92fc-8ed5-4646-8cb0-6ffb923869a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:37.959239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd92fc-8ed5-4646-8cb0-6ffb923869a2 service glance] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:37.984511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1144/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:38.092589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a4bb6-6601-482c-8aff-865e4dcda0ca None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.092957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a4bb6-6601-482c-8aff-865e4dcda0ca None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.093313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a4bb6-6601-482c-8aff-865e4dcda0ca None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.095031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1150/2294] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:38.100169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875e488-be69-448b-a87e-193c7498d509 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.100459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875e488-be69-448b-a87e-193c7498d509 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.100740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875e488-be69-448b-a87e-193c7498d509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.122952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d875e488-be69-448b-a87e-193c7498d509 None None] MFA Rules not processed for user `934be918dc8c4fa59988e5bb22e56eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:38.124183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d875e488-be69-448b-a87e-193c7498d509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:38.184976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1145/2295] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:38.198377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a4bb6-6601-482c-8aff-865e4dcda0ca None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:38.200217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0c3a4bb6-6601-482c-8aff-865e4dcda0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:38.213640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a4bb6-6601-482c-8aff-865e4dcda0ca None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:38.220472 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915b568-1cc8-4dc2-97e3-2452e7759c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:38.221260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b568-1cc8-4dc2-97e3-2452e7759c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.221636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b568-1cc8-4dc2-97e3-2452e7759c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.221978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b568-1cc8-4dc2-97e3-2452e7759c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.229627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915b568-1cc8-4dc2-97e3-2452e7759c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:38.231072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915b568-1cc8-4dc2-97e3-2452e7759c5f service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:38.238957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1151/2296] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:38.324908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85da114-3b2f-4671-962c-0cf09f29c78b None None] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.325215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85da114-3b2f-4671-962c-0cf09f29c78b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.325498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85da114-3b2f-4671-962c-0cf09f29c78b None None] PATH_INFO: `/` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.326928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1146/2297] 10.222.0.23 () {54 vars in 921 bytes} [Sat Sep 10 21:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:38.332507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae44f96-4e41-4086-87bb-54394d647bb0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.333142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae44f96-4e41-4086-87bb-54394d647bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.333142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae44f96-4e41-4086-87bb-54394d647bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.353513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eae44f96-4e41-4086-87bb-54394d647bb0 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:38.355665 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eae44f96-4e41-4086-87bb-54394d647bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:38.366027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1152/2298] 10.222.0.23 () {58 vars in 1018 bytes} [Sat Sep 10 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:38.379072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85da114-3b2f-4671-962c-0cf09f29c78b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:38.381247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b85da114-3b2f-4671-962c-0cf09f29c78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:38.399085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85da114-3b2f-4671-962c-0cf09f29c78b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:38.404749 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:38.405715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.406184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.406627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.418857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:38.421751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:38.438645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1147/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:45.007953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e54c2-0101-4050-9b8d-4b6d406fc88c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:45.008407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e54c2-0101-4050-9b8d-4b6d406fc88c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:45.008631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e54c2-0101-4050-9b8d-4b6d406fc88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:45.029302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7a3e54c2-0101-4050-9b8d-4b6d406fc88c None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:45.030644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7a3e54c2-0101-4050-9b8d-4b6d406fc88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:45.072017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:45.096065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1153/2300] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:45.110380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:45.111998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:45.198622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32858cf6-2a8a-4750-abca-eb469e1305c5 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:45.203572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:45.204191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:45.204469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:45.204742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:45.214072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:45.215753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:45.223858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1148/2301] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:02:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:02:55.086606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:55.220928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:02.038357 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97da22-65c0-45e7-857e-d8772bca882a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:02.038896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97da22-65c0-45e7-857e-d8772bca882a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:02.039184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97da22-65c0-45e7-857e-d8772bca882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:02.060552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2c97da22-65c0-45e7-857e-d8772bca882a None None] MFA Rules not processed for user `0e2ccb201b434349a8918e8094aa2950`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:02.061942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2c97da22-65c0-45e7-857e-d8772bca882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:02.127091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1154/2302] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:02.654531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:02.656179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:02.748322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363030c-a7c9-4d79-b7ce-29774de4f36c service swift] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:02.755986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:02.756865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:02.757229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:02.757613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:02.760034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:02.789752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:02.789752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:02.802035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1149/2303] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:06.070787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60226ccc-a64a-4891-869f-6d61527cd699 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.071273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60226ccc-a64a-4891-869f-6d61527cd699 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.071371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60226ccc-a64a-4891-869f-6d61527cd699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.095768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-60226ccc-a64a-4891-869f-6d61527cd699 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:06.097100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-60226ccc-a64a-4891-869f-6d61527cd699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:06.104993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1155/2304] 10.222.0.23 () {58 vars in 967 bytes} [Sat Sep 10 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:06.120532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.140537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8328a6b-e171-49bd-b207-ff533639b867 service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.145663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.146298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.146578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.146851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.156657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.158266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.166945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1150/2305] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:16.086994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=28 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:16.163743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=38 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:27.349926 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216602e0-6e56-40a1-af01-52c39664df17 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.350425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216602e0-6e56-40a1-af01-52c39664df17 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.350548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216602e0-6e56-40a1-af01-52c39664df17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.372148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-216602e0-6e56-40a1-af01-52c39664df17 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:27.373354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-216602e0-6e56-40a1-af01-52c39664df17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:27.380553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1156/2306] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:27.405051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:27.421652 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2b648-fd63-4a2a-ba5a-2f42be05e165 service swift] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:27.430320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:27.431087 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.431585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.431934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.440274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:27.441907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:27.450608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1151/2307] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:37.365331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:37.447221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:42.597668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:42.598123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:42.598214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:42.617615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:42.618785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:42.625653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1157/2308] 10.222.0.23 () {58 vars in 967 bytes} [Sat Sep 10 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:42.637863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:42.649058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682ddca-7512-4cd2-8613-f714ba720f44 service swift] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:42.654924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:42.655705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:42.655987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:42.656265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:42.664888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:42.666484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:42.674600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1152/2309] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:44.839988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:44.841483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:44.917800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f335-47b4-4271-80af-97df7568ce0c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:44.922572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:44.923278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:44.923584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:44.923875 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] PATH_INFO: `/v3/users/0e2ccb201b434349a8918e8094aa2950` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:44.925020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e2ccb201b434349a8918e8094aa2950)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:44.926961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:44.975996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e2ccb201b434349a8918e8094aa2950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:44.984424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e2ccb201b434349a8918e8094aa2950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:44.985080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e2ccb201b434349a8918e8094aa2950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:44.985703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e2ccb201b434349a8918e8094aa2950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:44.993125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a5249dcd-cd4f-40d3-b307-3c49dd29a8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e2ccb201b434349a8918e8094aa2950'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:45.011281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1158/2310] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:44 2022] DELETE /identity/v3/users/0e2ccb201b434349a8918e8094aa2950 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:45.189948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.201883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399f07d-e2c6-40a8-b661-7dd38ab8ee4a service swift] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.264697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.265399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.265830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.266167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] PATH_INFO: `/v3/projects/12e813ffd18d40b9be703ff655385397` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.267366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e813ffd18d40b9be703ff655385397)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.269381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.320696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e813ffd18d40b9be703ff655385397', 'request_id': 'req-0cb455d0-c84a-406c-ad28-bd2a92867616'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:45.328752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e813ffd18d40b9be703ff655385397', 'request_id': 'req-0cb455d0-c84a-406c-ad28-bd2a92867616'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:45.337589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e813ffd18d40b9be703ff655385397', 'request_id': 'req-0cb455d0-c84a-406c-ad28-bd2a92867616'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:45.349837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] The token cache is being invalidate because project 12e813ffd18d40b9be703ff655385397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:45.350129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:45.350778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0cb455d0-c84a-406c-ad28-bd2a92867616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:45.352907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1153/2311] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:45 2022] DELETE /identity/v3/projects/12e813ffd18d40b9be703ff655385397 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:45.370791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.371085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.371367 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.392440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:45.393709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.454849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1159/2312] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:45.463961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.464252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.464533 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.484445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:45.485717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.494223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1154/2313] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:45.498952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.508770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8584ab-7b1a-4e22-9d82-c4b3cb7ee7ef None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.513607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.514265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.514877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.514877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.515436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:45.515913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.518067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.527787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:45.532075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1160/2314] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:45.536753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.548017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9a9c-6f01-4b15-9c5c-26050b2b2d8d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.552697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.553301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.553619 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.553894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.554563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.557124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.587453 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1155/2315] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:45.591498 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.602791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e3f62-5d51-473d-a15c-0d2dad4efbe1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.665311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.665974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.666261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.666540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.667171 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.669102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.690947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1161/2316] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:45.694854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.705415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c77ad2-6aaf-43d8-b3fb-3bfeb604737c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.710041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.710664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.711638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.712036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.712692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.714340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.725430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1156/2317] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:45.730117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.740207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab572aa-2e92-4c54-b608-dcbf9eed1e52 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.745052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d9c34-33a4-4a92-995d-13d3583c3029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.745712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d9c34-33a4-4a92-995d-13d3583c3029 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.746002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d9c34-33a4-4a92-995d-13d3583c3029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.746278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d9c34-33a4-4a92-995d-13d3583c3029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.746921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d9c34-33a4-4a92-995d-13d3583c3029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.748522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d9c34-33a4-4a92-995d-13d3583c3029 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.760447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1162/2318] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:45.764676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.776106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbced43-7aa5-4e2c-8494-f73fef653e0c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.780644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.781294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.781616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.781888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] PATH_INFO: `/v3/projects/2acae87b2c2e4b8bad6bdc4e5015bfe1/users/2074e884f2534394b1d4273165d67e04/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.797249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2acae87b2c2e4b8bad6bdc4e5015bfe1, user_id=2074e884f2534394b1d4273165d67e04, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.799958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.813642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1157/2319] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:45 2022] PUT /identity/v3/projects/2acae87b2c2e4b8bad6bdc4e5015bfe1/users/2074e884f2534394b1d4273165d67e04/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:45.819133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9ff09-3df5-4d8e-8b06-3c1c4e433fee None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.819416 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9ff09-3df5-4d8e-8b06-3c1c4e433fee None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.819685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9ff09-3df5-4d8e-8b06-3c1c4e433fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.840772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6ef9ff09-3df5-4d8e-8b06-3c1c4e433fee None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:45.842058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6ef9ff09-3df5-4d8e-8b06-3c1c4e433fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.911000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1163/2320] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:45.920939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.922690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.939256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:45.985589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dda1a0-86bb-474f-a919-367e87ff1640 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.989168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c27d49-4868-470b-aaf5-d0f62b032eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.989816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c27d49-4868-470b-aaf5-d0f62b032eda service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.990101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c27d49-4868-470b-aaf5-d0f62b032eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.990377 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c27d49-4868-470b-aaf5-d0f62b032eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.999289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c27d49-4868-470b-aaf5-d0f62b032eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:46.000716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c27d49-4868-470b-aaf5-d0f62b032eda service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:46.010859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1158/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:50.575006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.575598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.575598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.606162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] MFA Rules not processed for user `2074e884f2534394b1d4273165d67e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:50.607384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:50.637442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.639284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.640266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.679518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] MFA Rules not processed for user `47f9883392364d3fb5d0b921d6f451d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:50.681178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:50.718307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1164/2322] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:50.727319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.739679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e2e42-2f74-4750-ac6e-55977385d894 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.745211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.746029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.746314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.746585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.747341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:50.749281 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:50.753838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1159/2323] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:50.759541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:50.789509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1165/2324] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:50.793015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.806148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706db6d-1a91-418d-a00f-d5f2571639d2 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.882265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.883605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.883898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.884325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.884997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:50.887044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:50.917516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1160/2325] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:03:50.919721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.932650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce86ba0-cae1-4e2e-a56e-d44f92a82c39 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.938334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.939436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.939854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.940381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.941153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:50.942864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:50.954301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1166/2326] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:03:50.958968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.971855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c27d6-b251-4dd6-9575-4c0ae858f35f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.978825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.979743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.980087 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.980547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] PATH_INFO: `/v3/projects/c4d0855b10794ef0abe8416633ab3afe/users/4a64fcc635fd43c495fca323a34f9d64/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.011295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d0855b10794ef0abe8416633ab3afe, user_id=4a64fcc635fd43c495fca323a34f9d64, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.012917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.031826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1161/2327] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:50 2022] PUT /identity/v3/projects/c4d0855b10794ef0abe8416633ab3afe/users/4a64fcc635fd43c495fca323a34f9d64/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:03:51.035914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.047249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07dbe-1a32-4904-a510-e8e989008f32 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.158023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f685cfa-109a-4519-9ad4-86725d775a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.158758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685cfa-109a-4519-9ad4-86725d775a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.159053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685cfa-109a-4519-9ad4-86725d775a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.159748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685cfa-109a-4519-9ad4-86725d775a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.160946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f685cfa-109a-4519-9ad4-86725d775a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.163049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f685cfa-109a-4519-9ad4-86725d775a61 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.183177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1167/2328] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:51.189633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.217397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b036-c5d2-433c-90ab-614741dbe6a6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.232084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.234051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.234051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.234051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] PATH_INFO: `/v3/projects/c4d0855b10794ef0abe8416633ab3afe/users/4a64fcc635fd43c495fca323a34f9d64/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.236003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d0855b10794ef0abe8416633ab3afe, user_id=4a64fcc635fd43c495fca323a34f9d64, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.238853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.268565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1162/2329] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:51 2022] PUT /identity/v3/projects/c4d0855b10794ef0abe8416633ab3afe/users/4a64fcc635fd43c495fca323a34f9d64/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:57.363064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2766eb7-f1be-4e5d-88c5-a883abdff673 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:57.363593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2766eb7-f1be-4e5d-88c5-a883abdff673 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:57.363687 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2766eb7-f1be-4e5d-88c5-a883abdff673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:57.436958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a2766eb7-f1be-4e5d-88c5-a883abdff673 None None] MFA Rules not processed for user `4a64fcc635fd43c495fca323a34f9d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:57.439109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a2766eb7-f1be-4e5d-88c5-a883abdff673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:57.668860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1168/2330] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:57.683988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:57.686516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:57.706641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:57.805149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b21c7a-4ad3-4b23-abcd-ca36027e66e7 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:57.812182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ec694-a48b-429e-b5e9-414f59d32c89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:57.813276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ec694-a48b-429e-b5e9-414f59d32c89 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:57.813886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ec694-a48b-429e-b5e9-414f59d32c89 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:57.814389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ec694-a48b-429e-b5e9-414f59d32c89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:57.908802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ec694-a48b-429e-b5e9-414f59d32c89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:57.910926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ec694-a48b-429e-b5e9-414f59d32c89 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:57.938186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1163/2331] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:59.478817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ff3fa-6f9c-4125-bd59-c262d1ced6bf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:59.479364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ff3fa-6f9c-4125-bd59-c262d1ced6bf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:59.479364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ff3fa-6f9c-4125-bd59-c262d1ced6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:59.500829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ce5ff3fa-6f9c-4125-bd59-c262d1ced6bf None None] MFA Rules not processed for user `2074e884f2534394b1d4273165d67e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:59.502085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ce5ff3fa-6f9c-4125-bd59-c262d1ced6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:59.509426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1169/2332] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:07.438476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448eb2a-529c-4c0d-9366-1e43cee4123b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:07.438950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448eb2a-529c-4c0d-9366-1e43cee4123b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:07.439089 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448eb2a-529c-4c0d-9366-1e43cee4123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:07.462370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7448eb2a-529c-4c0d-9366-1e43cee4123b None None] MFA Rules not processed for user `a22c6bd0fac8443c844d7734aba97ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:07.463665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7448eb2a-529c-4c0d-9366-1e43cee4123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:07.524241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1164/2333] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:09.187865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:09.188397 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:09.188511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:09.208740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] MFA Rules not processed for user `16610ae2c3214396b78fb1caff01ab6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:09.210774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:09.257431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:09.268898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1170/2334] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:17.517083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=75 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:19.267327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:37.315534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:37.316053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:37.316154 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:37.335904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] MFA Rules not processed for user `6e24c7d558f74ce181aad463bf18ee52`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:37.336948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:37.390430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1165/2335] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:04:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:47.383937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=25 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:48.670396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:48.724623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705bf30-75d7-4437-b120-81c3ba1028d0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:48.729054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:48.729688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:48.729969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:48.730244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:48.731948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:48.745087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:48.746550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:48.758093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1171/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:58.752800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=26 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:01.074765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.084228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b7c05-67f5-44eb-a7a1-e938be331d8c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.088579 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.089175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.089614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.089718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.091415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:01.155584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.156933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:01.167946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1166/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:04.493865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.496188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:04.515659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a91eb-8369-4f03-9bbd-4055f3b26c08 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.520313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.521257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.521603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.521912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] PATH_INFO: `/v3/users/47f9883392364d3fb5d0b921d6f451d7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.523062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f9883392364d3fb5d0b921d6f451d7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.525347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.574306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f9883392364d3fb5d0b921d6f451d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.581361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f9883392364d3fb5d0b921d6f451d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.582026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f9883392364d3fb5d0b921d6f451d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.582728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f9883392364d3fb5d0b921d6f451d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.589720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-aaf34ade-9c78-46b6-8c62-4d78010f32dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f9883392364d3fb5d0b921d6f451d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.607234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1172/2338] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:04 2022] DELETE /identity/v3/users/47f9883392364d3fb5d0b921d6f451d7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:04.784880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.795201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deda756-3590-476b-b40c-e6acf06245ac service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.853318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.853988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.854274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.854553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] PATH_INFO: `/v3/projects/12481b41ce3541eeb05a8a1722d9acbf` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.855670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12481b41ce3541eeb05a8a1722d9acbf)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.857653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.905840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12481b41ce3541eeb05a8a1722d9acbf', 'request_id': 'req-a9401e0e-f0fb-4679-9f93-7b86aba58754'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.912338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12481b41ce3541eeb05a8a1722d9acbf', 'request_id': 'req-a9401e0e-f0fb-4679-9f93-7b86aba58754'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.919992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12481b41ce3541eeb05a8a1722d9acbf', 'request_id': 'req-a9401e0e-f0fb-4679-9f93-7b86aba58754'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.931528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] The token cache is being invalidate because project 12481b41ce3541eeb05a8a1722d9acbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:04.931816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.932432 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a9401e0e-f0fb-4679-9f93-7b86aba58754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:04.934009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1167/2339] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:04 2022] DELETE /identity/v3/projects/12481b41ce3541eeb05a8a1722d9acbf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:04.939009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.939325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.939635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.960230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:04.961934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:05.020474 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1173/2340] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:05.028401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.028688 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.028962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.048951 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:05.050398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:05.059411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1168/2341] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:05.063254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.074142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a440-47ff-41ef-be26-3cbd6940b637 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.079217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.079827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.080113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.080388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.081033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:05.081344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.083078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.095496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1174/2342] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:05.099982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.110283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15025337-4b40-43bd-b9cf-c7d4646ca106 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.115324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.116051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.116324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.116597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.117252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.120150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.137501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:05.154633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1169/2343] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:05.158957 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.170233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cbcf0-06d2-46f9-8a8a-c8be56af71bd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.232312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.232924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.233207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.233510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.234143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.236088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.245328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:05.263906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1175/2344] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:05.268034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.278888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbf9e5-873e-4e57-b3d9-bc6d0cc1ba0d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.284529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.285370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.285715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.286002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.286890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.289356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.304514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1170/2345] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:05.306294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.318027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdec21-ddfd-4903-ad0d-83dd981aa198 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.324821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42078cc-3e6f-4b77-8528-34a6a67a0a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.325621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42078cc-3e6f-4b77-8528-34a6a67a0a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.325880 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42078cc-3e6f-4b77-8528-34a6a67a0a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.326211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42078cc-3e6f-4b77-8528-34a6a67a0a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.327100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42078cc-3e6f-4b77-8528-34a6a67a0a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.328931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42078cc-3e6f-4b77-8528-34a6a67a0a1d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.344107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1176/2346] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:05.348523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.360317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b0a9b-c982-4b30-b165-4c64123f239e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.366810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.367471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.367830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.368966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] PATH_INFO: `/v3/projects/23bdeebc37ab4c7dbbd5f6df98434494/users/9a724ed1f9164dc797119b6331dcc2ce/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.390192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23bdeebc37ab4c7dbbd5f6df98434494, user_id=9a724ed1f9164dc797119b6331dcc2ce, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.393058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.406963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1171/2347] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:05 2022] PUT /identity/v3/projects/23bdeebc37ab4c7dbbd5f6df98434494/users/9a724ed1f9164dc797119b6331dcc2ce/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:05.420117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.420328 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.420606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.452134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] MFA Rules not processed for user `9a724ed1f9164dc797119b6331dcc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:05.453386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:05.526413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1177/2348] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:05.530724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.594854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4beeb8-5303-42e7-b652-7588f6fbbfa8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.599525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.600134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.600417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.600693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.601331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.604043 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.630434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1172/2349] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:05.634551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.645101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f6491-a6ef-4a86-845a-8136f46c9095 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.702648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.703176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.703451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.703720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.704348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.706283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.728438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1178/2350] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:05.732543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.743426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c2cb-cdf9-442e-9d65-727284b3b558 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.748035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.748645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.749006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.749289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.749975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.751581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.762235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1173/2351] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:05.766334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.778132 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d42cc-a445-41b4-9d6b-99bfb9539423 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.782626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.783271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.783545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.783818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] PATH_INFO: `/v3/projects/9f1b6a0b9ad144bf9c227215be399ed0/users/a26c479caeef4afbbdeef80007ae9233/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.799658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1b6a0b9ad144bf9c227215be399ed0, user_id=a26c479caeef4afbbdeef80007ae9233, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.802407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.820568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1179/2352] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:05 2022] PUT /identity/v3/projects/9f1b6a0b9ad144bf9c227215be399ed0/users/a26c479caeef4afbbdeef80007ae9233/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:05.825379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.840452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa9433-29e8-4408-ba05-4bc67e9f484b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.903710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28360f93-e36c-4051-9033-8677583658f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.904326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28360f93-e36c-4051-9033-8677583658f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.904611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28360f93-e36c-4051-9033-8677583658f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.904896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28360f93-e36c-4051-9033-8677583658f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.905570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28360f93-e36c-4051-9033-8677583658f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.907166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28360f93-e36c-4051-9033-8677583658f8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.917658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1174/2353] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:05.921938 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.932868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237bb1-ff6c-4467-880d-bd4a461d4e63 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.937423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.938096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.938378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.938652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] PATH_INFO: `/v3/projects/9f1b6a0b9ad144bf9c227215be399ed0/users/a26c479caeef4afbbdeef80007ae9233/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.940537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1b6a0b9ad144bf9c227215be399ed0, user_id=a26c479caeef4afbbdeef80007ae9233, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.943222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.953407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1180/2354] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:05 2022] PUT /identity/v3/projects/9f1b6a0b9ad144bf9c227215be399ed0/users/a26c479caeef4afbbdeef80007ae9233/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:05.963431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.963719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.963990 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.992546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] MFA Rules not processed for user `a26c479caeef4afbbdeef80007ae9233`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:05.994372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:06.013318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:06.064588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1175/2355] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:06.076963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:06.078686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:06.100896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:06.137943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbc3b-5bb0-4e17-a01b-55056a340e66 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:06.141542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:06.142141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:06.142422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:06.142699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:06.151849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:06.153263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:06.162483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1181/2356] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:06.244889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:06.246389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:06.258581 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd223e-6fa5-4eff-a014-3e12ae526d7b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:06.262404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:06.263005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:06.263294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:06.263589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:06.322200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:06.323910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:06.332635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1176/2357] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:09.409837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:09.418333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957843c7-5ccf-40c1-9fa6-770a3a0a90dd service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:09.422145 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:09.422742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:09.423024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:09.423300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:09.424998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:09.483625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:09.485096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65608b-fdf5-456f-98f7-4a3fc1c75996 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:09.494731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1182/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:09.543897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:09.545322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:09.618783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddfa7-95df-41e6-8661-835ab4d8f4d9 service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:09.624156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:09.624754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:09.625034 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:09.625307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:09.627058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:09.675671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:09.686064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:09.687640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:09.696669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1177/2359] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:19.490502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=45 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:19.692521 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:20.380569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:20.381225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:20.381605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:20.417905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:20.419121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:20.489758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1183/2360] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:20.573302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:20.575481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:20.719468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfba9f-3d33-4fb3-b0c5-924a84a9d700 service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:20.738323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e1e1-9852-4a6a-8fe3-f1082e6bdf66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:20.748827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e1e1-9852-4a6a-8fe3-f1082e6bdf66 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:20.749225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e1e1-9852-4a6a-8fe3-f1082e6bdf66 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:20.750367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e1e1-9852-4a6a-8fe3-f1082e6bdf66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:20.778329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e1e1-9852-4a6a-8fe3-f1082e6bdf66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:20.781284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e1e1-9852-4a6a-8fe3-f1082e6bdf66 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:20.807855 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1178/2361] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:05:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:05:21.121384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:21.124233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:21.262306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76874df-bfca-4858-b891-ec2f647c6ec1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:21.275390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:21.276045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:21.276326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:21.276603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:21.282870 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:21.312976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:21.314632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:21.335155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1184/2362] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:23.700790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:23.701970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:23.702509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:23.723076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] MFA Rules not processed for user `16610ae2c3214396b78fb1caff01ab6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:23.724947 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:23.738553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1179/2363] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:24.300628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:24.302227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:24.443866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f1777-cb66-4fc7-9968-f9d02a3a8086 service placement] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:24.458736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:24.459390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:24.460974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:24.463848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:24.467062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:24.483740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:24.485492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:24.504024 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1185/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:25.012713 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:25.028926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07487-cfee-4573-b50f-48c1e8c8588e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:25.036016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:25.036931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:25.037388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:25.037853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:25.040787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:25.054364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:25.056927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:25.069735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1180/2365] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:26.953584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:26.955112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:27.039223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:27.075085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d87e-8a9e-41f8-95ba-8b703c25f808 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:27.090356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:27.091043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:27.091621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:27.092019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] PATH_INFO: `/v3/users/2074e884f2534394b1d4273165d67e04` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:27.093371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2074e884f2534394b1d4273165d67e04)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:27.095615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:27.183811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2074e884f2534394b1d4273165d67e04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.192847 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2074e884f2534394b1d4273165d67e04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.196490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2074e884f2534394b1d4273165d67e04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.196490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2074e884f2534394b1d4273165d67e04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.204523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2074e884f2534394b1d4273165d67e04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.233105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1186/2366] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:26 2022] DELETE /identity/v3/users/2074e884f2534394b1d4273165d67e04 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:27.519029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:27.535198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6439ce-4965-4492-ba1d-685fa796022f service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:27.675021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:27.675983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:27.676653 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:27.677023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] PATH_INFO: `/v3/projects/2acae87b2c2e4b8bad6bdc4e5015bfe1` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:27.678372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2acae87b2c2e4b8bad6bdc4e5015bfe1)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:27.681161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:27.760457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2acae87b2c2e4b8bad6bdc4e5015bfe1', 'request_id': 'req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.769076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2acae87b2c2e4b8bad6bdc4e5015bfe1', 'request_id': 'req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.793115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2acae87b2c2e4b8bad6bdc4e5015bfe1', 'request_id': 'req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.805716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:27.811631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] The token cache is being invalidate because project 2acae87b2c2e4b8bad6bdc4e5015bfe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:27.811913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.814748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:27.821500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1181/2367] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:27 2022] DELETE /identity/v3/projects/2acae87b2c2e4b8bad6bdc4e5015bfe1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:27.825554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:27.827388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:27.848062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bb2cd-e35c-40d9-bfb6-fdf68f6aece1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:27.958905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:27.959614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:27.959827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:27.960102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] PATH_INFO: `/v3/users/4a64fcc635fd43c495fca323a34f9d64` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:27.962519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a64fcc635fd43c495fca323a34f9d64)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:27.964592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:28.071810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a64fcc635fd43c495fca323a34f9d64'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.091196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a64fcc635fd43c495fca323a34f9d64'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.092031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a64fcc635fd43c495fca323a34f9d64'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.092808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a64fcc635fd43c495fca323a34f9d64'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.105063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:28.111228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a64fcc635fd43c495fca323a34f9d64'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.131934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:28.134517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:28.152207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1187/2368] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:27 2022] DELETE /identity/v3/users/4a64fcc635fd43c495fca323a34f9d64 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:28.272254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87d5c-d8e2-42fd-a4bc-e956d3e64403 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:28.371705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:28.372563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:28.373033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:28.373305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:28.380861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:28.402797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:28.404267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:28.424473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1182/2369] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:28.720922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:28.720922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:28.801857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b970-fa66-434e-afbf-fca46e487f4e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:28.808271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:28.808915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:28.809209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:28.809502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] PATH_INFO: `/v3/projects/c4d0855b10794ef0abe8416633ab3afe` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:28.810647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d0855b10794ef0abe8416633ab3afe)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:28.812614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:28.878696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d0855b10794ef0abe8416633ab3afe', 'request_id': 'req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.891683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d0855b10794ef0abe8416633ab3afe', 'request_id': 'req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.912767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d0855b10794ef0abe8416633ab3afe', 'request_id': 'req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.931212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] The token cache is being invalidate because project c4d0855b10794ef0abe8416633ab3afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:28.931827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.932821 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:28.935067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1188/2370] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:28 2022] DELETE /identity/v3/projects/c4d0855b10794ef0abe8416633ab3afe => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:32.361560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:32.363176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:32.463313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2670a-3c80-4abc-8be3-f683209cb891 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:32.527111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:32.545920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:32.547203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:32.547491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:32.547934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:32.549751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:32.625274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:32.626852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:32.638104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1183/2371] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:38.927541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=60,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:40.166123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.167663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:40.187068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f4ac2-671f-4ce9-8f47-e310fe10e6a8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.196774 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.198086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.198792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.198792 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.201012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:40.215995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.217846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799be1b-1ea7-4cf8-b578-9b212b2c6c28 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.230528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1189/2372] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:42.632451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=30 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:43.145957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:43.157862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629759f4-c705-4cd4-8b6c-8b51aa23b88d service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:43.163508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:43.164751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:43.165097 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:43.165510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:43.168011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:43.180477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:43.182197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:43.192796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1184/2373] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:46.501260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:46.501750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:46.501950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:46.521686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:46.522930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:46.588345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1190/2374] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:46.602827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:46.604402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:46.712605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa393f83-23d7-4b55-9f98-962f8a98d171 service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:46.720701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9b595-35a0-468c-b43e-c18a963d8aa0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:46.721331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b595-35a0-468c-b43e-c18a963d8aa0 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:46.721938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b595-35a0-468c-b43e-c18a963d8aa0 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:46.722197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b595-35a0-468c-b43e-c18a963d8aa0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:46.732733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9b595-35a0-468c-b43e-c18a963d8aa0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:46.734340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9b595-35a0-468c-b43e-c18a963d8aa0 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:46.746215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1185/2375] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:05:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:05:47.938872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:47.940386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.069528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc56091-6508-402b-8f7a-9f16d25f45f4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:48.078324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:48.079181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:48.079633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:48.080077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] PATH_INFO: `/v3/users/9a724ed1f9164dc797119b6331dcc2ce` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:48.081555 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a724ed1f9164dc797119b6331dcc2ce)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:48.083664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:48.171670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a724ed1f9164dc797119b6331dcc2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.185912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a724ed1f9164dc797119b6331dcc2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.188073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a724ed1f9164dc797119b6331dcc2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.189214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a724ed1f9164dc797119b6331dcc2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.202895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a724ed1f9164dc797119b6331dcc2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.249894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1191/2376] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:47 2022] DELETE /identity/v3/users/9a724ed1f9164dc797119b6331dcc2ce => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:48.254704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:48.255310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:48.255310 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:48.286830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:48.288070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.360670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1186/2377] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:48.372173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:48.373824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.389513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:48.445061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b034da-2ebf-474a-841f-53f2ee287615 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:48.449012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:48.449662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:48.449945 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:48.450219 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:48.459983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:48.461996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:48.474225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1192/2378] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:48.796518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:48.798252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.822351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8f05e-de09-417d-beee-09c9b170aea1 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:48.827923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:48.828645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:48.829109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:48.829732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] PATH_INFO: `/v3/projects/23bdeebc37ab4c7dbbd5f6df98434494` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:48.831597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23bdeebc37ab4c7dbbd5f6df98434494)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:48.843866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:48.913110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23bdeebc37ab4c7dbbd5f6df98434494', 'request_id': 'req-c4650db1-8a44-4388-bb75-5d1a9c41b98f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.922585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23bdeebc37ab4c7dbbd5f6df98434494', 'request_id': 'req-c4650db1-8a44-4388-bb75-5d1a9c41b98f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.932644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23bdeebc37ab4c7dbbd5f6df98434494', 'request_id': 'req-c4650db1-8a44-4388-bb75-5d1a9c41b98f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.952471 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:48.962960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] The token cache is being invalidate because project 23bdeebc37ab4c7dbbd5f6df98434494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:48.963509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.964596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.966372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1187/2379] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:48 2022] DELETE /identity/v3/projects/23bdeebc37ab4c7dbbd5f6df98434494 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:48.970862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:48.983660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cded1-57f5-4b24-8ebd-3d29e29ff81e service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.052438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.053007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.053337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.053597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] PATH_INFO: `/v3/users/a26c479caeef4afbbdeef80007ae9233` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.056991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26c479caeef4afbbdeef80007ae9233)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.059051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.125866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26c479caeef4afbbdeef80007ae9233'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.138500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26c479caeef4afbbdeef80007ae9233'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.139260 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26c479caeef4afbbdeef80007ae9233'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.140156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26c479caeef4afbbdeef80007ae9233'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.161538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b97a3d5-686f-4587-a1b0-30cb0d1e2aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26c479caeef4afbbdeef80007ae9233'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.186199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1193/2380] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:48 2022] DELETE /identity/v3/users/a26c479caeef4afbbdeef80007ae9233 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:49.510502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.511970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:49.534139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4650db1-8a44-4388-bb75-5d1a9c41b98f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.605646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.606291 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.606623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.606900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] PATH_INFO: `/v3/projects/9f1b6a0b9ad144bf9c227215be399ed0` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.608077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1b6a0b9ad144bf9c227215be399ed0)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.612864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.674654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1b6a0b9ad144bf9c227215be399ed0', 'request_id': 'req-84fd2091-7561-4624-820b-6a00ce2b83ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.686143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1b6a0b9ad144bf9c227215be399ed0', 'request_id': 'req-84fd2091-7561-4624-820b-6a00ce2b83ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.697819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1b6a0b9ad144bf9c227215be399ed0', 'request_id': 'req-84fd2091-7561-4624-820b-6a00ce2b83ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.709335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] The token cache is being invalidate because project 9f1b6a0b9ad144bf9c227215be399ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:49.709867 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.711220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-84fd2091-7561-4624-820b-6a00ce2b83ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.712442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1188/2381] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:49 2022] DELETE /identity/v3/projects/9f1b6a0b9ad144bf9c227215be399ed0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:49.717494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.717809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.718107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.738254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:49.739469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:49.778166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:49.821548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1194/2382] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:49.824443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.824728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.825006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.846816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:49.848326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:49.860094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1189/2383] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:49.861122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.874064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33117339-866e-40df-b84e-5fcd67ec2628 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.879276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.879904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.880184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.880461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.881121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:49.881435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.884190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.909496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1195/2384] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:49.914099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.924026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c7bd-a005-44ea-b1c2-62341c197d1d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.929384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.929829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.930107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.930380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.931003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.932902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.968678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1190/2385] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/projects => generated 357 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:49.973079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.984079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121272fb-6731-4a48-8472-8f8f23ce7a30 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.046184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.046895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.047185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.047465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.048113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.050496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.081534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1196/2386] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/users => generated 279 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:50.084986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.100465 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443f36f-c10a-43c3-bee3-340b15fe1103 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.106762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.107468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.107751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.108172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.108880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.110610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.122525 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1191/2387] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:50.127329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.138937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3c91-26ab-4404-a790-e98e321f3696 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.143630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da9b95-e11c-4bd5-a53f-b47d74855674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.144238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da9b95-e11c-4bd5-a53f-b47d74855674 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.144518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da9b95-e11c-4bd5-a53f-b47d74855674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.144790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da9b95-e11c-4bd5-a53f-b47d74855674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.145721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da9b95-e11c-4bd5-a53f-b47d74855674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.147373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da9b95-e11c-4bd5-a53f-b47d74855674 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.172950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1197/2388] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:50.174980 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.189924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a976e5-d156-47da-a516-28b462df7112 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.196700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.197791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.198232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.198664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] PATH_INFO: `/v3/projects/349e541a8bfd48548251e44915f52890/users/f6091452ef7d4bc58fd3abfc3934d098/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.223203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349e541a8bfd48548251e44915f52890, user_id=f6091452ef7d4bc58fd3abfc3934d098, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.226221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.240732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1192/2389] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:50 2022] PUT /identity/v3/projects/349e541a8bfd48548251e44915f52890/users/f6091452ef7d4bc58fd3abfc3934d098/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:50.252312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.252598 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.252869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.285395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] MFA Rules not processed for user `f6091452ef7d4bc58fd3abfc3934d098`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:50.286972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:50.365191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1198/2390] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:50.370685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.406075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:50.464861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd558e2c-c0ed-4bc0-b71a-0769d08b3d12 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.470024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.470651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.470932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.471209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.471850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.473839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.506673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1193/2391] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/projects => generated 355 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:50.508477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.525561 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3065a4-e1c9-4b56-9ee0-cc4a004150d1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.557020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:50.600004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.601890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.602181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.602486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.603161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.605131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.631765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1199/2392] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/users => generated 284 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:50.636833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.648069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63f814-e5f7-4d5b-94db-47f4e3025cfc admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.653066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.653722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.654013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.654293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.654948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.656564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.668760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1194/2393] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:50.673104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.686883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c67495-baf4-43c3-b779-d437c4f661d1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.691760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.692427 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.692708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.692980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] PATH_INFO: `/v3/projects/a1634df8a50643d6b5db69be75c569e2/users/d19e308859a943fba16c99bb38670c3f/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.712053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1634df8a50643d6b5db69be75c569e2, user_id=d19e308859a943fba16c99bb38670c3f, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.715638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.730689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1200/2394] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:50 2022] PUT /identity/v3/projects/a1634df8a50643d6b5db69be75c569e2/users/d19e308859a943fba16c99bb38670c3f/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:50.734390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.746360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f104191-4c53-4774-9084-4e40fff42418 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.810917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56adfb-a332-4819-9904-163b71a5ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.811712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56adfb-a332-4819-9904-163b71a5ad86 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.812102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56adfb-a332-4819-9904-163b71a5ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.812492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56adfb-a332-4819-9904-163b71a5ad86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.813237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56adfb-a332-4819-9904-163b71a5ad86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.815158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56adfb-a332-4819-9904-163b71a5ad86 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.832614 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1195/2395] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:50.833915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.846814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc009d-b87d-4878-ac30-e611c903f663 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.855110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.856081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.856467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.856907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] PATH_INFO: `/v3/projects/a1634df8a50643d6b5db69be75c569e2/users/d19e308859a943fba16c99bb38670c3f/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.859227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1634df8a50643d6b5db69be75c569e2, user_id=d19e308859a943fba16c99bb38670c3f, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.871019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.893247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1201/2396] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:50 2022] PUT /identity/v3/projects/a1634df8a50643d6b5db69be75c569e2/users/d19e308859a943fba16c99bb38670c3f/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:50.903184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.903473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.903751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.982939 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] MFA Rules not processed for user `d19e308859a943fba16c99bb38670c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:50.984509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.078983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1196/2397] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:51.091675 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.093394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.174969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963d7f-20af-467a-b7e9-c92c243056c2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.180432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cea2b6-4c0d-4f2f-b75b-e8171b5ee70a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.180722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cea2b6-4c0d-4f2f-b75b-e8171b5ee70a service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.180842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cea2b6-4c0d-4f2f-b75b-e8171b5ee70a service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.181019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cea2b6-4c0d-4f2f-b75b-e8171b5ee70a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.192271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cea2b6-4c0d-4f2f-b75b-e8171b5ee70a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.193748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cea2b6-4c0d-4f2f-b75b-e8171b5ee70a service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.214172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1202/2398] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:51.306224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.317396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52b19b-b22c-4f9b-b61e-1b5e328f4d58 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.322288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.322901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.323181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.323454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.348288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:51.394158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.396246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.408718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1197/2399] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:01.201699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:01.325310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.325770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.325959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.347913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:01.349604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:01.401930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=18 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:01.406809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1203/2400] 10.222.0.23 () {58 vars in 967 bytes} [Sat Sep 10 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:01.420137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.421564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:01.492301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ff9c7-1b8e-4402-bfca-d314b8e7921c service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.497349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceac07-bda6-4e03-874e-7c7d653454a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.498014 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceac07-bda6-4e03-874e-7c7d653454a7 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.498298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceac07-bda6-4e03-874e-7c7d653454a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.498570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceac07-bda6-4e03-874e-7c7d653454a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.506551 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeceac07-bda6-4e03-874e-7c7d653454a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.508140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeceac07-bda6-4e03-874e-7c7d653454a7 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.516191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1198/2401] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:06:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:06:06.554951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:06.617421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f10e1e-edfa-4e6c-85e6-4195e942a6e6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.621971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.622657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.622905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.623160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] PATH_INFO: `/v3/users/f6091452ef7d4bc58fd3abfc3934d098` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.624270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6091452ef7d4bc58fd3abfc3934d098)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.626234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.681335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6091452ef7d4bc58fd3abfc3934d098'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.689104 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6091452ef7d4bc58fd3abfc3934d098'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.689796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6091452ef7d4bc58fd3abfc3934d098'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.690428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6091452ef7d4bc58fd3abfc3934d098'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.698111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6091452ef7d4bc58fd3abfc3934d098'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.716293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1204/2402] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:06 2022] DELETE /identity/v3/users/f6091452ef7d4bc58fd3abfc3934d098 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:06.720763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.721229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.721324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.740758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:06.742397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.807343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1199/2403] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:06.817030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:06.818466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.852383 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:06.884058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b083f-4346-4fe5-b89e-aac3452a9572 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.887630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.888238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.888521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.888795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.898432 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.899879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.909388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1205/2404] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:07.201853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.203361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:07.216418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72171c-042a-47a8-ad43-f03231be1605 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.221446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.222113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.222399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.222740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] PATH_INFO: `/v3/projects/349e541a8bfd48548251e44915f52890` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.223868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349e541a8bfd48548251e44915f52890)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.225852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.281854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349e541a8bfd48548251e44915f52890', 'request_id': 'req-505d3d76-e724-4431-b735-6ab3c80d2ba8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.288553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349e541a8bfd48548251e44915f52890', 'request_id': 'req-505d3d76-e724-4431-b735-6ab3c80d2ba8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.296113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349e541a8bfd48548251e44915f52890', 'request_id': 'req-505d3d76-e724-4431-b735-6ab3c80d2ba8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.308599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] The token cache is being invalidate because project 349e541a8bfd48548251e44915f52890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:07.308891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.309589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.311290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1200/2405] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:07 2022] DELETE /identity/v3/projects/349e541a8bfd48548251e44915f52890 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:07.315419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.326674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bb14c-33b7-4b57-96b0-a3d7ac95420c service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.387087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.387729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.388006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.388278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] PATH_INFO: `/v3/users/d19e308859a943fba16c99bb38670c3f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.389389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19e308859a943fba16c99bb38670c3f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.391326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.439552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19e308859a943fba16c99bb38670c3f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.447963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19e308859a943fba16c99bb38670c3f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.448622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19e308859a943fba16c99bb38670c3f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.449326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19e308859a943fba16c99bb38670c3f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.456537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-63add36f-a077-498e-9285-0530ac1d5015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19e308859a943fba16c99bb38670c3f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.474867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1206/2406] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:07 2022] DELETE /identity/v3/users/d19e308859a943fba16c99bb38670c3f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:07.764775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.766301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:07.779916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d3d76-e724-4431-b735-6ab3c80d2ba8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.788902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:07.854255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.855229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.855511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.855844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] PATH_INFO: `/v3/projects/a1634df8a50643d6b5db69be75c569e2` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.856967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1634df8a50643d6b5db69be75c569e2)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.858907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.910728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1634df8a50643d6b5db69be75c569e2', 'request_id': 'req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.917422 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1634df8a50643d6b5db69be75c569e2', 'request_id': 'req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.924985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1634df8a50643d6b5db69be75c569e2', 'request_id': 'req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.936058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] The token cache is being invalidate because project a1634df8a50643d6b5db69be75c569e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:07.936290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.936917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-7c60b537-0dbc-44a5-b8bb-f2c20ee5532e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:07.938436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1201/2407] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:07 2022] DELETE /identity/v3/projects/a1634df8a50643d6b5db69be75c569e2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:07.944276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.945326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.945423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.965449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:07.967023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:08.008698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:08.028100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1207/2408] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:08.036923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.037229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.037542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.058613 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:08.059828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:08.067699 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1202/2409] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:08.072674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.082215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623573-6940-425e-8eb8-feb22e55dc13 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.086846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.087435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.087712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.088012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.088888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:08.089020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.090609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.102154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1208/2410] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:08.106486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.115607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d53f2-f47a-4c6d-8ea7-deef619e2ed2 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.120748 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.121335 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.121646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.121920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.122537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.124406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.154721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1203/2411] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:08.156209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.167072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d67060-4333-4ece-886d-39a84d117d18 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.222878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.223401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.223682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.223951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.224602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.227103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.250222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1209/2412] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:08.253786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.264590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f8478-41c8-41a3-8730-d635e663fd74 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.269132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.270013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.270292 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.270560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.271199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.272843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.283487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1204/2413] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:08.287431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.298161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507416a-48cd-4345-8fce-44c498ab05fe admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.302843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ed2e-9814-4f26-a7ec-039912e11357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.303439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ed2e-9814-4f26-a7ec-039912e11357 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.304749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ed2e-9814-4f26-a7ec-039912e11357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.305053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ed2e-9814-4f26-a7ec-039912e11357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.305735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8ed2e-9814-4f26-a7ec-039912e11357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.307341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8ed2e-9814-4f26-a7ec-039912e11357 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.318037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1210/2414] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:08.323180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.333131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad0386-0306-4253-86a9-35b2c4d0a220 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.338519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.339159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.339439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.339712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] PATH_INFO: `/v3/projects/88e54e10e6d845dd99f83700ba5ceb8c/users/b1212df1a3e84cd8886ebd73b11d5c78/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.354927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e54e10e6d845dd99f83700ba5ceb8c, user_id=b1212df1a3e84cd8886ebd73b11d5c78, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.357616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.371528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1205/2415] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:08 2022] PUT /identity/v3/projects/88e54e10e6d845dd99f83700ba5ceb8c/users/b1212df1a3e84cd8886ebd73b11d5c78/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:08.381852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.382143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.382419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.410400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] MFA Rules not processed for user `b1212df1a3e84cd8886ebd73b11d5c78`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:08.411608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:08.476170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1211/2416] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:08.480608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.524195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:08.544919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d7324-fb34-4b09-96ab-3fd657fe1536 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.549481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.550175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.550451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.550723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.551354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.553240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.579907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1206/2417] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:08.583987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.594730 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87d60b-5bae-498e-bd4d-0ee1de2c1413 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.635550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:08.653627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.654236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.654520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.654796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.655428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.657363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.677713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1212/2418] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/users => generated 307 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:08.682466 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.693761 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5e193-952f-4692-918b-2efe79d0740e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.698969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.699574 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.699851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.700124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.700759 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.702397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.713100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1207/2419] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:08.717209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.727177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e3390-7ba2-42f9-95d2-608deab87170 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.731660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.732302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.732577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.732846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] PATH_INFO: `/v3/projects/6d15882f2ccf4ae5b8bf0e06b1e235f5/users/f71cf0699af344248edc3bc40597a596/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.747704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d15882f2ccf4ae5b8bf0e06b1e235f5, user_id=f71cf0699af344248edc3bc40597a596, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.750388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.763199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1213/2420] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:08 2022] PUT /identity/v3/projects/6d15882f2ccf4ae5b8bf0e06b1e235f5/users/f71cf0699af344248edc3bc40597a596/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:08.767156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.777953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91255551-256a-4b09-a305-5a3eca7f942b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.833882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb76b8-a135-42f2-8a91-2badb178fdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.834475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb76b8-a135-42f2-8a91-2badb178fdc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.834781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb76b8-a135-42f2-8a91-2badb178fdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.835086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb76b8-a135-42f2-8a91-2badb178fdc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.835877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb76b8-a135-42f2-8a91-2badb178fdc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.837455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb76b8-a135-42f2-8a91-2badb178fdc0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.847017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1208/2421] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:08.851185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.861290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d215d-35ce-4883-a55c-087ea36b7f3b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.865834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.867046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.867321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.867592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] PATH_INFO: `/v3/projects/6d15882f2ccf4ae5b8bf0e06b1e235f5/users/f71cf0699af344248edc3bc40597a596/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.869442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d15882f2ccf4ae5b8bf0e06b1e235f5, user_id=f71cf0699af344248edc3bc40597a596, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.872094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.882708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1214/2422] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:08 2022] PUT /identity/v3/projects/6d15882f2ccf4ae5b8bf0e06b1e235f5/users/f71cf0699af344248edc3bc40597a596/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:08.893484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.893789 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.894068 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.925086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] MFA Rules not processed for user `f71cf0699af344248edc3bc40597a596`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:08.926317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:08.997546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1209/2423] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:09.008181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.010288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:09.071158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2bd6f-8d99-43cd-b2be-a9b503b165d7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.074794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:09.075398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.075676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.075948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.084740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:09.086465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.096306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1215/2424] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:09.175279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.185098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8212fb-d26a-46da-b132-edc8f74cc3a8 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.193920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:09.195499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.195803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.196079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.228838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:09.254806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:09.256169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.265828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1210/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:10.890609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:10.892301 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:11.026336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427e97-2159-46ea-aab0-9b0f9d7c713a service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:11.032076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:11.032685 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.032963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.033238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.035844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:11.055363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:11.057283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528febd-bff5-4cf0-934d-d8953a70e94e service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:11.072144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1216/2426] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:13.442864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:13.509250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c7b0-562e-40ac-918f-a2b62488d52d service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:13.513856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:13.514480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:13.514760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:13.515033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] PATH_INFO: `/v3/users/b1212df1a3e84cd8886ebd73b11d5c78` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:13.516202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1212df1a3e84cd8886ebd73b11d5c78)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:13.518352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:13.567253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1212df1a3e84cd8886ebd73b11d5c78'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.574837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1212df1a3e84cd8886ebd73b11d5c78'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.575519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1212df1a3e84cd8886ebd73b11d5c78'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.583742 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1212df1a3e84cd8886ebd73b11d5c78'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.596167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1212df1a3e84cd8886ebd73b11d5c78'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.602380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1211/2427] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:13 2022] DELETE /identity/v3/users/b1212df1a3e84cd8886ebd73b11d5c78 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:13.606859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:13.607336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:13.607442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:13.628079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:13.629398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.646733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:13.689989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1217/2428] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:13.700278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:13.701712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.760475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a6241-b2d2-4257-8e5c-a5b9fa17e84b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:13.764129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:13.764768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:13.765053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:13.765326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:13.773306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:13.776426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:13.778506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:13.791609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1212/2429] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:14.069545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.071176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:14.083936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded6b0c-cf94-42ec-b429-464c9261d5b6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.089914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.090570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.090849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.091123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] PATH_INFO: `/v3/projects/88e54e10e6d845dd99f83700ba5ceb8c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.092347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e54e10e6d845dd99f83700ba5ceb8c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.094250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.142810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e54e10e6d845dd99f83700ba5ceb8c', 'request_id': 'req-0e7bd425-566d-413f-b12b-ccb7409e92b4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.149837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e54e10e6d845dd99f83700ba5ceb8c', 'request_id': 'req-0e7bd425-566d-413f-b12b-ccb7409e92b4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.157379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e54e10e6d845dd99f83700ba5ceb8c', 'request_id': 'req-0e7bd425-566d-413f-b12b-ccb7409e92b4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.170278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] The token cache is being invalidate because project 88e54e10e6d845dd99f83700ba5ceb8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:14.170573 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.171200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.172817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1218/2430] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:14 2022] DELETE /identity/v3/projects/88e54e10e6d845dd99f83700ba5ceb8c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:14.176810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.187681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55039a4b-b271-465e-b1a2-3a0593808bd3 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.244565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.245195 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.245495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.245771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] PATH_INFO: `/v3/users/f71cf0699af344248edc3bc40597a596` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.246880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71cf0699af344248edc3bc40597a596)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.248780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.294829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71cf0699af344248edc3bc40597a596'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.302264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71cf0699af344248edc3bc40597a596'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.302912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71cf0699af344248edc3bc40597a596'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.309872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71cf0699af344248edc3bc40597a596'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.321502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1e23fef7-23ad-46b5-9587-59b47e829981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71cf0699af344248edc3bc40597a596'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.327247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1213/2431] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:14 2022] DELETE /identity/v3/users/f71cf0699af344248edc3bc40597a596 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:14.602718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.604246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:14.619473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bd425-566d-413f-b12b-ccb7409e92b4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.679748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.680577 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.680969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.681348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] PATH_INFO: `/v3/projects/6d15882f2ccf4ae5b8bf0e06b1e235f5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.688368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d15882f2ccf4ae5b8bf0e06b1e235f5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.690708 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.739959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:14.741616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d15882f2ccf4ae5b8bf0e06b1e235f5', 'request_id': 'req-917c72de-dc0b-4391-97d1-87357a19a476'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.749761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d15882f2ccf4ae5b8bf0e06b1e235f5', 'request_id': 'req-917c72de-dc0b-4391-97d1-87357a19a476'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.759040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d15882f2ccf4ae5b8bf0e06b1e235f5', 'request_id': 'req-917c72de-dc0b-4391-97d1-87357a19a476'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.770217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] The token cache is being invalidate because project 6d15882f2ccf4ae5b8bf0e06b1e235f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:14.770505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.771134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-917c72de-dc0b-4391-97d1-87357a19a476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.773317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1219/2432] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:14 2022] DELETE /identity/v3/projects/6d15882f2ccf4ae5b8bf0e06b1e235f5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:14.778276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.778733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.778846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.798798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:14.799929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:14.859284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1214/2433] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:14.867248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.867533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.867806 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.890741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:14.891942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:14.901864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1220/2434] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:14.905916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.915790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f01e2f-7b31-4ba7-9286-42d63e8f2859 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.920697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.921309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.921634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.921913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.922570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:14.922876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.924628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.937256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1215/2435] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:14.941276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.951110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe2559-d80b-499b-b362-ddadd0612bfc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.955662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.956275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.956552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.956823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.957449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.959421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.985733 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1221/2436] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:14 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:14.989851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.995666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:15.001989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9345-c21c-4baf-954a-d0a0335d3103 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.060160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.060694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.060978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.061253 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.061915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.063837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.085030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1216/2437] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:14 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:15.089192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.101365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e0a7-0cf3-4bbe-855a-ef7302e8127a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.106178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.106784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.107065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.107336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.107974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.109612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.120988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1222/2438] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:15.124937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.136288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fca9fa-fbac-471c-ac91-a05e6c1eab29 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.140891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3907-436d-48d5-b1d3-2996d8f83d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.141508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3907-436d-48d5-b1d3-2996d8f83d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.141823 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3907-436d-48d5-b1d3-2996d8f83d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.142101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3907-436d-48d5-b1d3-2996d8f83d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.142740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3907-436d-48d5-b1d3-2996d8f83d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.144338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3907-436d-48d5-b1d3-2996d8f83d98 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.155712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1217/2439] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:15.159834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.169948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c856c-77e8-437e-a57b-e19fdb700efb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.174584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.175228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.175511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.175811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] PATH_INFO: `/v3/projects/8fcfc90edd354d469921864da0c4b6d1/users/79cef8b7fcb3442390b5754ad7e92bad/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.191869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcfc90edd354d469921864da0c4b6d1, user_id=79cef8b7fcb3442390b5754ad7e92bad, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.195179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.207843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1223/2440] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:15 2022] PUT /identity/v3/projects/8fcfc90edd354d469921864da0c4b6d1/users/79cef8b7fcb3442390b5754ad7e92bad/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:15.219482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.219777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.220058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.248890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] MFA Rules not processed for user `79cef8b7fcb3442390b5754ad7e92bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:15.250143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.314879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1218/2441] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:15.320372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.383202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ba49-3b52-4853-8985-4ef3bc85472b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.387858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.388457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.388737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.389010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.389655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.391557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.418300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1224/2442] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:15.422341 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.433669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe92af-6f9c-4ce8-a989-cb99fc4ed8f8 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.490313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.491502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.492009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.492287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.493076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.495081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.519098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:15.526084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1219/2443] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:15.530098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.546967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bb866-51d6-4e2a-b9e4-46881ca67473 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.551659 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.552264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.552546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.552818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.554149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.555672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.566694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1225/2444] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:15.570900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.583625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4fbf9-d8b3-4163-9647-dc556318c036 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.590952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.591710 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.592100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.592449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] PATH_INFO: `/v3/projects/0c9b4bc49a224d0c88b781af98240bff/users/4bd42076cc0540149195f64562f1af1c/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.609022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9b4bc49a224d0c88b781af98240bff, user_id=4bd42076cc0540149195f64562f1af1c, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.611837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.627556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1220/2445] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:15 2022] PUT /identity/v3/projects/0c9b4bc49a224d0c88b781af98240bff/users/4bd42076cc0540149195f64562f1af1c/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:15.632540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.642865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43170c53-ba7d-4960-9e4b-c53918a96054 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.700630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:15.715914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257fd70-6345-4156-9070-df3009a935aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.716522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fd70-6345-4156-9070-df3009a935aa admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.716797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fd70-6345-4156-9070-df3009a935aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.717067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fd70-6345-4156-9070-df3009a935aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.717732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257fd70-6345-4156-9070-df3009a935aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.719312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257fd70-6345-4156-9070-df3009a935aa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.729824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1226/2446] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:15.733925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.744281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fb6a7-0566-4899-b1d7-694609d91067 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.748857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.749537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.749824 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.750102 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] PATH_INFO: `/v3/projects/0c9b4bc49a224d0c88b781af98240bff/users/4bd42076cc0540149195f64562f1af1c/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.752048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9b4bc49a224d0c88b781af98240bff, user_id=4bd42076cc0540149195f64562f1af1c, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.754739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.764755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1221/2447] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:15 2022] PUT /identity/v3/projects/0c9b4bc49a224d0c88b781af98240bff/users/4bd42076cc0540149195f64562f1af1c/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:15.775537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.775851 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.776124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.804236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] MFA Rules not processed for user `4bd42076cc0540149195f64562f1af1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:15.805420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.872787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1227/2448] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:15.883943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.885863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.946146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff774d-8036-482f-8549-820a99719713 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.949791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d622fe8-844c-4f63-b9f5-2b562795beb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.950389 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d622fe8-844c-4f63-b9f5-2b562795beb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.950671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d622fe8-844c-4f63-b9f5-2b562795beb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.950946 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d622fe8-844c-4f63-b9f5-2b562795beb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.962611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d622fe8-844c-4f63-b9f5-2b562795beb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.964064 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d622fe8-844c-4f63-b9f5-2b562795beb1 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.974267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1222/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:16.112308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.174658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f23dc5-e72b-4b62-abbb-7760dfdc0745 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.179040 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.179658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.179934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.180204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] PATH_INFO: `/v3/users/79cef8b7fcb3442390b5754ad7e92bad` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.181299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cef8b7fcb3442390b5754ad7e92bad)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.183246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.228623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cef8b7fcb3442390b5754ad7e92bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.235955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cef8b7fcb3442390b5754ad7e92bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.236564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cef8b7fcb3442390b5754ad7e92bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.237183 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cef8b7fcb3442390b5754ad7e92bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.244258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cef8b7fcb3442390b5754ad7e92bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.250118 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:16.264131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1228/2450] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:16 2022] DELETE /identity/v3/users/79cef8b7fcb3442390b5754ad7e92bad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:16.268640 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.269007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.269358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.292899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:16.294268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.355332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1223/2451] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:16.364933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.366520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.423971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75db6f-aa94-4410-892a-3b704eaa4e6b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.427619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.428165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.429036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.429316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.438225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.439636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.449667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1229/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:16.720510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.724174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.736902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dcd9d-05c5-43d3-b57c-8bbc52777d33 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.742548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.743304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.744018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.744416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] PATH_INFO: `/v3/projects/8fcfc90edd354d469921864da0c4b6d1` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.745930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcfc90edd354d469921864da0c4b6d1)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.748445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.806655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.807756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcfc90edd354d469921864da0c4b6d1', 'request_id': 'req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.808087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.814372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcfc90edd354d469921864da0c4b6d1', 'request_id': 'req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.819690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f6bb2-1f30-432a-9892-3e475192f4f3 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.824148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcfc90edd354d469921864da0c4b6d1', 'request_id': 'req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.825453 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:16.838963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] The token cache is being invalidate because project 8fcfc90edd354d469921864da0c4b6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:16.839255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.840384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.842036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1224/2453] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:16 2022] DELETE /identity/v3/projects/8fcfc90edd354d469921864da0c4b6d1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:16.845802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.847131 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.874037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.874650 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.874932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.875241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.877100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.918481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea5cf-9533-4d65-910d-1c5813d8bc6a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.923205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.923911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.924187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.924460 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] PATH_INFO: `/v3/users/4bd42076cc0540149195f64562f1af1c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.925599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd42076cc0540149195f64562f1af1c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.927522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.950670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.952044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.963386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1230/2454] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:16.977171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd42076cc0540149195f64562f1af1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.984785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd42076cc0540149195f64562f1af1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.985427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd42076cc0540149195f64562f1af1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.998611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd42076cc0540149195f64562f1af1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.015388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-145f47b4-8341-428c-9fc3-0511295139c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd42076cc0540149195f64562f1af1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.022167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1225/2455] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:16 2022] DELETE /identity/v3/users/4bd42076cc0540149195f64562f1af1c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:17.298159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.311869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc377057-71f1-4f32-a191-c5f4b422e06d service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.355905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:17.376677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.377635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.377921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.378597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] PATH_INFO: `/v3/projects/0c9b4bc49a224d0c88b781af98240bff` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.379393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9b4bc49a224d0c88b781af98240bff)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.381374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.433163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9b4bc49a224d0c88b781af98240bff', 'request_id': 'req-08780fa2-171f-4344-a2e7-eabf3fb21e36'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.439547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9b4bc49a224d0c88b781af98240bff', 'request_id': 'req-08780fa2-171f-4344-a2e7-eabf3fb21e36'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.447019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9b4bc49a224d0c88b781af98240bff', 'request_id': 'req-08780fa2-171f-4344-a2e7-eabf3fb21e36'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.458290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] The token cache is being invalidate because project 0c9b4bc49a224d0c88b781af98240bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:17.458540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.459238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-08780fa2-171f-4344-a2e7-eabf3fb21e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.461677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1231/2456] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:17 2022] DELETE /identity/v3/projects/0c9b4bc49a224d0c88b781af98240bff => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:17.466594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.466877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.467151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.487935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:17.489139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:17.551252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1226/2457] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:17.559196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e6b95-1482-444b-a5db-44decf981259 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.559484 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e6b95-1482-444b-a5db-44decf981259 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.559762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e6b95-1482-444b-a5db-44decf981259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.580175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-059e6b95-1482-444b-a5db-44decf981259 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:17.581393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-059e6b95-1482-444b-a5db-44decf981259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:17.591169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1232/2458] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:17.595252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.605160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65541-6a17-4a57-9c05-85cb47cc44ea None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.610053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.610732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.611101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.611451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.612174 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:17.612487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.614127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.616703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:17.628351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1227/2459] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:17.633310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e6b95-1482-444b-a5db-44decf981259 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.642871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e6b95-1482-444b-a5db-44decf981259 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.647936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.648534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.648808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.649074 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.649739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.651621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.682244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1233/2460] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:17.686524 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.696977 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113bca6-e438-48d5-8e4f-45f80c7c7de1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.756724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.757259 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.757576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.757858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.758504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.760456 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.785974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1228/2461] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:17.787564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.798406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7ef54-3cc4-4c41-8380-127bc6c222a8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.802986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.804297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.804576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.804848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.805497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.807096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.817389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1234/2462] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:17.821218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.831962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb554-f06e-4c07-9c4e-0c4af33628cb admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.837338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c19271-6d05-4d70-bf0e-3b7489307824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.838011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19271-6d05-4d70-bf0e-3b7489307824 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.838300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19271-6d05-4d70-bf0e-3b7489307824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.838610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19271-6d05-4d70-bf0e-3b7489307824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.839242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c19271-6d05-4d70-bf0e-3b7489307824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.840883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c19271-6d05-4d70-bf0e-3b7489307824 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.853308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1229/2463] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:17.863623 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.879772 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe73399-44ad-4177-b6e4-426cd4b6f121 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.886921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.887828 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.888268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.888554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] PATH_INFO: `/v3/projects/99809958216a43e6956098be23412d51/users/7dfadbe82312491ea73dd47764d3d592/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.908109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99809958216a43e6956098be23412d51, user_id=7dfadbe82312491ea73dd47764d3d592, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.911312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.933569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1235/2464] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:17 2022] PUT /identity/v3/projects/99809958216a43e6956098be23412d51/users/7dfadbe82312491ea73dd47764d3d592/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:17.996977 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb245043-254d-45e6-a480-d135e206401b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.997557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb245043-254d-45e6-a480-d135e206401b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.997557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb245043-254d-45e6-a480-d135e206401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.035958 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-bb245043-254d-45e6-a480-d135e206401b None None] MFA Rules not processed for user `7dfadbe82312491ea73dd47764d3d592`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:18.037226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bb245043-254d-45e6-a480-d135e206401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.111065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1230/2465] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:18.116247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.179222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae97a-c5ce-4998-a17d-b9130a237998 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.184006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.184613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.184889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.185159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.185841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.187737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.200390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:18.219453 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1236/2466] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:18 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:18.223572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb245043-254d-45e6-a480-d135e206401b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.235124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb245043-254d-45e6-a480-d135e206401b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.296981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.298017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.298519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.299633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.300817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.305140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.315644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:18.331020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1231/2467] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:18 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:18.332227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.342763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6332843-a4c1-4582-b621-7314a49d1149 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.347431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.348031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.348761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.348761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.349204 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.351355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.361973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1237/2468] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:18.366817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.381865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c217c9-6657-46a3-b5f8-acfa90d886ee admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.386859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.387568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.387810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.388153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] PATH_INFO: `/v3/projects/9fec0364c54c480eaae3e5bc06f995fb/users/6f5473ecefeb48388ce11ebd8946d9c1/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.405229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fec0364c54c480eaae3e5bc06f995fb, user_id=6f5473ecefeb48388ce11ebd8946d9c1, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.407954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.424827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1232/2469] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:18 2022] PUT /identity/v3/projects/9fec0364c54c480eaae3e5bc06f995fb/users/6f5473ecefeb48388ce11ebd8946d9c1/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:18.429916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.440860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca011a0-7c99-4ae1-8b78-c6d5d8169369 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.507283 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d9047-3c90-49f2-a6ef-5824e92ff036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.507981 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d9047-3c90-49f2-a6ef-5824e92ff036 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.508336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d9047-3c90-49f2-a6ef-5824e92ff036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.508796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d9047-3c90-49f2-a6ef-5824e92ff036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.509562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d9047-3c90-49f2-a6ef-5824e92ff036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.511262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d9047-3c90-49f2-a6ef-5824e92ff036 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.522939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1238/2470] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:18.527482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.538861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14791b18-2f4a-44e5-9ba2-bd0a6a49a5ea admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.543498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.544151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.544431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.544735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] PATH_INFO: `/v3/projects/9fec0364c54c480eaae3e5bc06f995fb/users/6f5473ecefeb48388ce11ebd8946d9c1/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.546656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fec0364c54c480eaae3e5bc06f995fb, user_id=6f5473ecefeb48388ce11ebd8946d9c1, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.549402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.564323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1233/2471] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:18 2022] PUT /identity/v3/projects/9fec0364c54c480eaae3e5bc06f995fb/users/6f5473ecefeb48388ce11ebd8946d9c1/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:18.575010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.575948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.576245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.610196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] MFA Rules not processed for user `6f5473ecefeb48388ce11ebd8946d9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:18.611434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.689327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1239/2472] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:18.701986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.703664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.770025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2535c16-37cb-43dd-bfa1-8f8be6d95e3e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.774375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5bcd0-dfc6-452c-9df5-11f930d06fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.774989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5bcd0-dfc6-452c-9df5-11f930d06fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.775272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5bcd0-dfc6-452c-9df5-11f930d06fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.775549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5bcd0-dfc6-452c-9df5-11f930d06fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.785220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5bcd0-dfc6-452c-9df5-11f930d06fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.786723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5bcd0-dfc6-452c-9df5-11f930d06fc7 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.798240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1234/2473] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:19.270615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.350510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37cbcf-5a04-4a5b-ab00-9cbc0d7c31d8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.356165 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.356862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.357128 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.357725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] PATH_INFO: `/v3/users/7dfadbe82312491ea73dd47764d3d592` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.358920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfadbe82312491ea73dd47764d3d592)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.361060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.365811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:19.420105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfadbe82312491ea73dd47764d3d592'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.430327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfadbe82312491ea73dd47764d3d592'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.430970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfadbe82312491ea73dd47764d3d592'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.431575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfadbe82312491ea73dd47764d3d592'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.439299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfadbe82312491ea73dd47764d3d592'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.463207 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1240/2474] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:19 2022] DELETE /identity/v3/users/7dfadbe82312491ea73dd47764d3d592 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:19.467817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.468315 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.468436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.494612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:19.495931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.612675 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1235/2475] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:19.623612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.625048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.773593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b1038-f071-4a7a-9bf4-bea9ce714841 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.791496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.792131 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.792421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.792691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.805633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.807444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.823166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1241/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:20.126712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.128134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.206634 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.208744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.221608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:20.235936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b555ed-7e2e-4688-a81c-4fcbf55ebef0 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.242649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b407386-3ef9-4eba-8338-b53c5eb802b4 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.247884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.248587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.248930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.249177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] PATH_INFO: `/v3/projects/99809958216a43e6956098be23412d51` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.251062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99809958216a43e6956098be23412d51)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.252666 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.253112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.253301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.253626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.253907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.255900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.270544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.272408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.285753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1236/2477] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:20.286900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.288841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.307191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8f8e2-4982-4eb3-98c7-f944a98dea3d service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.313107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99809958216a43e6956098be23412d51', 'request_id': 'req-42412da9-c60e-45d9-a8af-3a9354bdd009'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.321576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99809958216a43e6956098be23412d51', 'request_id': 'req-42412da9-c60e-45d9-a8af-3a9354bdd009'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.331195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99809958216a43e6956098be23412d51', 'request_id': 'req-42412da9-c60e-45d9-a8af-3a9354bdd009'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.351361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] The token cache is being invalidate because project 99809958216a43e6956098be23412d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:20.351684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.352338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.354100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1242/2478] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:20 2022] DELETE /identity/v3/projects/99809958216a43e6956098be23412d51 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:20.358026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.359445 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.376172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:20.376902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.377661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.378011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.378428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] PATH_INFO: `/v3/users/6e24c7d558f74ce181aad463bf18ee52` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.379672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e24c7d558f74ce181aad463bf18ee52)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.382781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.435309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e24c7d558f74ce181aad463bf18ee52'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.443133 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e24c7d558f74ce181aad463bf18ee52'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.444570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412da9-c60e-45d9-a8af-3a9354bdd009 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.445096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e24c7d558f74ce181aad463bf18ee52'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.449777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.450410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.450686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.450962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] PATH_INFO: `/v3/users/6f5473ecefeb48388ce11ebd8946d9c1` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.452098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5473ecefeb48388ce11ebd8946d9c1)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.453344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e24c7d558f74ce181aad463bf18ee52'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.454063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.470695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e24c7d558f74ce181aad463bf18ee52'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.477871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1237/2479] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:20 2022] DELETE /identity/v3/users/6e24c7d558f74ce181aad463bf18ee52 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:20.510576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5473ecefeb48388ce11ebd8946d9c1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.518784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5473ecefeb48388ce11ebd8946d9c1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.519782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5473ecefeb48388ce11ebd8946d9c1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.520544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5473ecefeb48388ce11ebd8946d9c1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.528853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5473ecefeb48388ce11ebd8946d9c1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.547250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1243/2480] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:20 2022] DELETE /identity/v3/users/6f5473ecefeb48388ce11ebd8946d9c1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:20.667051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.668775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.755106 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771560b-8f8b-4013-949f-eb706c2c3b05 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.768776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:20.822316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.823028 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.823618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.823618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] PATH_INFO: `/v3/projects/c38e86c82baa44028fd2dfafbb49f42c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.824837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38e86c82baa44028fd2dfafbb49f42c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.827084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.860293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.861272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.877477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db6402-7c34-45b3-a5c8-5f9997081758 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.885389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.886111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.886578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.887538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] PATH_INFO: `/v3/projects/9fec0364c54c480eaae3e5bc06f995fb` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.888586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fec0364c54c480eaae3e5bc06f995fb)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.890857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.909565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38e86c82baa44028fd2dfafbb49f42c', 'request_id': 'req-fba2586f-8592-4208-9539-73851740e626'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.916527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38e86c82baa44028fd2dfafbb49f42c', 'request_id': 'req-fba2586f-8592-4208-9539-73851740e626'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.926741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38e86c82baa44028fd2dfafbb49f42c', 'request_id': 'req-fba2586f-8592-4208-9539-73851740e626'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.939117 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] The token cache is being invalidate because project c38e86c82baa44028fd2dfafbb49f42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:20.939408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.940063 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-fba2586f-8592-4208-9539-73851740e626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.941599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1238/2481] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:20 2022] DELETE /identity/v3/projects/c38e86c82baa44028fd2dfafbb49f42c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:20.947039 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4609f-d00c-43a7-a481-244ff8167205 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.947326 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4609f-d00c-43a7-a481-244ff8167205 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.947605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4609f-d00c-43a7-a481-244ff8167205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.954228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fec0364c54c480eaae3e5bc06f995fb', 'request_id': 'req-613ee072-c203-492d-9d2e-d549e7a4a0e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.961810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fec0364c54c480eaae3e5bc06f995fb', 'request_id': 'req-613ee072-c203-492d-9d2e-d549e7a4a0e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.970067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7cb4609f-d00c-43a7-a481-244ff8167205 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:20.970411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fec0364c54c480eaae3e5bc06f995fb', 'request_id': 'req-613ee072-c203-492d-9d2e-d549e7a4a0e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.971294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7cb4609f-d00c-43a7-a481-244ff8167205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.982710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] The token cache is being invalidate because project 9fec0364c54c480eaae3e5bc06f995fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:20.983736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.984629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-613ee072-c203-492d-9d2e-d549e7a4a0e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:20.986640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1244/2482] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:20 2022] DELETE /identity/v3/projects/9fec0364c54c480eaae3e5bc06f995fb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:20.993179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.993501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.993807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.022406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.023741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.028315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.057816 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1239/2483] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:21.065985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5884-f057-4d92-88da-b75b8a228fd7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.066360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5884-f057-4d92-88da-b75b8a228fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.066642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5884-f057-4d92-88da-b75b8a228fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.090795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-002c5884-f057-4d92-88da-b75b8a228fd7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.091756 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-002c5884-f057-4d92-88da-b75b8a228fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.097558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1245/2484] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:21.100165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1240/2485] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:21.104442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.104728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.104978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.105293 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.119240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06392d49-535d-4c60-9226-c0f232d0aa30 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.126261 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.127012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.127320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.127594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.127709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.128250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:21.128560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.128886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.130372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.149190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1241/2486] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:21.154964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1246/2487] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:21.158661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.159469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.169196 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ea9bf-e24c-45c0-bed8-d4f111dbb2ea None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.173779 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7831-e515-4815-87d4-d3d8306f758c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.174234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.174843 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.175121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.175398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.176101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:21.176398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.178053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.181241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.182343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.182631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.182910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.183548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.185564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.197450 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1242/2488] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:21.198535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.211552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287fa52-fad1-4f8a-8f90-9c16034b17c9 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.220935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1247/2489] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:21.221606 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.222526 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.223069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.223617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.224501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.224744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.226853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.252615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.262366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1243/2490] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/projects => generated 359 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:21.266352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.298767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07a1a-a438-4361-9165-4d4b3f3cbdd9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.340794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa339a-ac72-419c-a3b5-2f65b5ef3fc3 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.346385 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.347100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.347401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.347664 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.348298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.350369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.363628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.364242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.364536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.364812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.366009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.368458 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.377639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1244/2491] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/users => generated 280 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:21.379668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.391711 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876719b0-eee4-47a1-9921-ccda5ea4e3d6 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.396676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1248/2492] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:21.396814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.397784 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.398038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.398305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.398706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.398945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.400544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.408403 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216d76a-8320-43cd-952f-82390a79f8a9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.414542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1245/2493] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:21.414678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.414678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.414678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.414678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.415424 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.415592 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.416955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.422400 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.427021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb0857-a05d-4b89-a82a-7716981fa38e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.431143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1249/2494] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:21.431938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.432417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.432595 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.432877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.433175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.433859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.435462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.446233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daa327-7fde-436f-b644-28b9b67790c4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.449495 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1246/2495] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:21.451366 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.451654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.451958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.452236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.452504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.453158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.454809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.461812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54104c-0bc7-43fe-bac3-bf070f880d31 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.466356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1250/2496] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:21.469650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633c248-1e64-47a6-90cd-6c2aaa9c2718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.470302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.470436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633c248-1e64-47a6-90cd-6c2aaa9c2718 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.470588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633c248-1e64-47a6-90cd-6c2aaa9c2718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.470866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633c248-1e64-47a6-90cd-6c2aaa9c2718 admin admin] PATH_INFO: `/v3/projects/bf3061f7fd6b4039bc466140d4f837d1/users/e639aa0b639c46f28d7ef61e516b3cd8/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.481491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65daf-ba42-4036-8705-e4221b8743a5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.486198 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633c248-1e64-47a6-90cd-6c2aaa9c2718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3061f7fd6b4039bc466140d4f837d1, user_id=e639aa0b639c46f28d7ef61e516b3cd8, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.486934 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357b9a9-551e-42d3-9c53-bc671cf5bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.487540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b9a9-551e-42d3-9c53-bc671cf5bf67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.487987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b9a9-551e-42d3-9c53-bc671cf5bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.488268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357b9a9-551e-42d3-9c53-bc671cf5bf67 admin admin] PATH_INFO: `/v3/projects/fb4ba3c342a44899a0085770cf2f2aae/users/9fe75898e3b04ec0b1e17257654f7bc8/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.488875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633c248-1e64-47a6-90cd-6c2aaa9c2718 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.502857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1247/2497] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:21 2022] PUT /identity/v3/projects/bf3061f7fd6b4039bc466140d4f837d1/users/e639aa0b639c46f28d7ef61e516b3cd8/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:21.505296 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357b9a9-551e-42d3-9c53-bc671cf5bf67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4ba3c342a44899a0085770cf2f2aae, user_id=9fe75898e3b04ec0b1e17257654f7bc8, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.508525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357b9a9-551e-42d3-9c53-bc671cf5bf67 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.513858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.514229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.514634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.522478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1251/2498] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:21 2022] PUT /identity/v3/projects/fb4ba3c342a44899a0085770cf2f2aae/users/9fe75898e3b04ec0b1e17257654f7bc8/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:21.533735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.534019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.534294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.546577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] MFA Rules not processed for user `e639aa0b639c46f28d7ef61e516b3cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.547787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.562682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] MFA Rules not processed for user `9fe75898e3b04ec0b1e17257654f7bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.563874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.620298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1248/2499] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:21.636715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1252/2500] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:21.645509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.655215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.656636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.681344 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.713333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdddc18-573b-48bf-babe-fa42a1844c15 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.718862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.719175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a261c1-833d-4f93-a173-fc2ec3579046 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.719682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.720160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.720443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.721120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.737269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.737652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.737652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.737652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.737652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.743674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.745185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.755590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1253/2501] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:21.765505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1249/2502] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:21.769066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.772973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.782142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2e27-21ff-4989-9c64-b95f67b06a39 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.841654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534aa02-eeae-4dd1-abde-d83cbd7a4925 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.841883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.847359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.848084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.848372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.848651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] PATH_INFO: `/v3/users/e639aa0b639c46f28d7ef61e516b3cd8` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.849825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e639aa0b639c46f28d7ef61e516b3cd8)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.849977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.850944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.851224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.851497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.851940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.852133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.854547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.882903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1254/2503] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:21.883883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.894853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505fbf5-f24a-49e3-b117-733ba6f23468 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.899819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.900425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.900704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.900975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.902048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.903235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.908825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e639aa0b639c46f28d7ef61e516b3cd8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.913513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1255/2504] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:21.918058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e639aa0b639c46f28d7ef61e516b3cd8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.918725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e639aa0b639c46f28d7ef61e516b3cd8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.918843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.920564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.926075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e639aa0b639c46f28d7ef61e516b3cd8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.934217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f743b7b-a33e-48a0-a74e-71b7faf5f23a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.939038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.939203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9d8c8106-7fd1-4336-b343-95b66ba2873b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e639aa0b639c46f28d7ef61e516b3cd8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.939641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.939921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.940191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.940818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.942433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.945840 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1250/2505] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:21 2022] DELETE /identity/v3/users/e639aa0b639c46f28d7ef61e516b3cd8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:21.950132 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.950417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.950690 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.957199 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1256/2506] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:21.961781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.970457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.971325 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721a53b-201f-430a-8ac8-ab9a79257ff9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.971646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.002643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:22.031608 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03568d02-0953-44be-93d3-bffcee68a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.032264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03568d02-0953-44be-93d3-bffcee68a889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.032540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03568d02-0953-44be-93d3-bffcee68a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.032814 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03568d02-0953-44be-93d3-bffcee68a889 admin admin] PATH_INFO: `/v3/projects/b6a44ccbf8274f74b971ff27523c9025/users/87aed76ac6174efd8e1d2ffb6087363d/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.036522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1251/2507] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:22.046922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.048294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.048434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03568d02-0953-44be-93d3-bffcee68a889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a44ccbf8274f74b971ff27523c9025, user_id=87aed76ac6174efd8e1d2ffb6087363d, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.051140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03568d02-0953-44be-93d3-bffcee68a889 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.065745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1257/2508] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:21 2022] PUT /identity/v3/projects/b6a44ccbf8274f74b971ff27523c9025/users/87aed76ac6174efd8e1d2ffb6087363d/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:22.076586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.076874 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.077151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.105641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] MFA Rules not processed for user `87aed76ac6174efd8e1d2ffb6087363d`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:22.106846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.108120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af61d2e-e816-49d5-b18d-9b05d9db3356 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.159425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.159848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.160135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.160412 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.177644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1258/2509] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:22.182080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.227570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.229000 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.233800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:22.252289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1252/2510] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:22.265083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf5443-bf7d-4e85-9dae-040a1d81f329 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.271905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.273050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.273645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.273936 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.274736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.276654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.315064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1259/2511] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:22.316624 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.332053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00a87f-4a8f-4c41-961b-211354b97986 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.398308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:22.419149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.419692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.419972 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.420830 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.421549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.423482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.454923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1253/2512] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:22.458512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.473429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83fa42-5a6b-4de8-9678-467734465f7c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.479518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.480268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.480622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.480941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.481882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.483844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.497513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1260/2513] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:22.501643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.513221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610e727-69cb-478c-bac5-db0dddcbcf1c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.518827 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.519744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.520084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.520341 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] PATH_INFO: `/v3/projects/48c18e61d9aa4dba95d33cdbba21b172/users/958737f7fe3b4523ae7a94c0e28003ba/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.538144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c18e61d9aa4dba95d33cdbba21b172, user_id=958737f7fe3b4523ae7a94c0e28003ba, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.541630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.559284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1254/2514] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:22 2022] PUT /identity/v3/projects/48c18e61d9aa4dba95d33cdbba21b172/users/958737f7fe3b4523ae7a94c0e28003ba/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:22.563439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.574470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.576295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.577238 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aaaa3-fcf3-45b1-8207-b24eb93f9a71 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.639560 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.640170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.640448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.640716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.641341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.642966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.652403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67333a-8860-40e8-86bc-0b93b6ccbdba admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.653315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1261/2515] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:22.657446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.660515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.661158 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.661448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.661760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] PATH_INFO: `/v3/projects/bf3061f7fd6b4039bc466140d4f837d1` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.662971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3061f7fd6b4039bc466140d4f837d1)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.664884 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.667337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff548913-b143-42d3-867d-d3a5f9a8ea75 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.671896 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736763b-6a44-44d7-ab89-f50f66c42cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.672468 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736763b-6a44-44d7-ab89-f50f66c42cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.672744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736763b-6a44-44d7-ab89-f50f66c42cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.673016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736763b-6a44-44d7-ab89-f50f66c42cd2 admin admin] PATH_INFO: `/v3/projects/48c18e61d9aa4dba95d33cdbba21b172/users/958737f7fe3b4523ae7a94c0e28003ba/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.674894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736763b-6a44-44d7-ab89-f50f66c42cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c18e61d9aa4dba95d33cdbba21b172, user_id=958737f7fe3b4523ae7a94c0e28003ba, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.677558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736763b-6a44-44d7-ab89-f50f66c42cd2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.691073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1262/2516] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:22 2022] PUT /identity/v3/projects/48c18e61d9aa4dba95d33cdbba21b172/users/958737f7fe3b4523ae7a94c0e28003ba/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:22.702540 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.702929 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.703211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.720993 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3061f7fd6b4039bc466140d4f837d1', 'request_id': 'req-9cdf5bd5-74ed-4ed6-989e-821d24423e28'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.727631 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3061f7fd6b4039bc466140d4f837d1', 'request_id': 'req-9cdf5bd5-74ed-4ed6-989e-821d24423e28'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.732839 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] MFA Rules not processed for user `958737f7fe3b4523ae7a94c0e28003ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:22.734407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.735602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3061f7fd6b4039bc466140d4f837d1', 'request_id': 'req-9cdf5bd5-74ed-4ed6-989e-821d24423e28'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.747282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] The token cache is being invalidate because project bf3061f7fd6b4039bc466140d4f837d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:22.747580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.748234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9cdf5bd5-74ed-4ed6-989e-821d24423e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.750012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1255/2517] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:22 2022] DELETE /identity/v3/projects/bf3061f7fd6b4039bc466140d4f837d1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:22.754435 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dd5ca-9a30-4264-82b4-66a116b8e407 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.754725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dd5ca-9a30-4264-82b4-66a116b8e407 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.755012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dd5ca-9a30-4264-82b4-66a116b8e407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.775296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4f7dd5ca-9a30-4264-82b4-66a116b8e407 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:22.776520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4f7dd5ca-9a30-4264-82b4-66a116b8e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.807112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1263/2518] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:22.814861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:22.820038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.821413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.842876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:22.844009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1256/2519] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:22.852926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.853350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.853678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.874648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:22.875898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.884635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1257/2520] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:22.887431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964a6aa-ed9b-44f8-8d68-d0b0ab99b60a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.889018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.891153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.891763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.892039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.892306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.898578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96ad8f-9a3b-4c67-b626-f3c10bd9c6dd None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.903272 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.903771 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.903902 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.904171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.904448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.905100 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:22.905251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.905439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.907115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.916977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1264/2521] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:22.919512 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1258/2522] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:22.924002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.934853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238b62f-a17f-48fa-89b1-8febae596051 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.939833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.940535 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.940800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.941124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.941786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.943739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.966104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.968071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.974041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1265/2523] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:22.979013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.981639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60397-6d75-4275-adb9-360f36a78e62 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.032421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.033072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.033350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.033651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.035439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:23.040852 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0ca8-aaf8-4d15-bedd-d90cdcf20ad3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.045558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.046518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.046518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.046741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.047317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.049240 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.074670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1266/2524] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:23.075706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.085310 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc89d1e-e549-4c68-8517-f5025a78f8dd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.090840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.091459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.091739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.092010 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.092639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.094307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.097859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.099340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.105910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1267/2525] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:23.109907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.110837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1259/2526] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:23.120663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71303515-10ff-4171-a92a-3c8998cce5ea admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.125294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fe474-bb20-46a5-8736-5f956cb4826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.125940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fe474-bb20-46a5-8736-5f956cb4826b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.126202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fe474-bb20-46a5-8736-5f956cb4826b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.126476 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fe474-bb20-46a5-8736-5f956cb4826b admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.127232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fe474-bb20-46a5-8736-5f956cb4826b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.128721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fe474-bb20-46a5-8736-5f956cb4826b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.140037 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1268/2527] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:23.143945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.156273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cff9a-f36f-429f-a093-f73abe933cfe service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.161542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.162165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.162600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.163556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] PATH_INFO: `/v3/projects/15dd70cd3158499996e666026e293811/users/b86d6ec3c196460c9c4e2fa4713fb2b7/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.180632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15dd70cd3158499996e666026e293811, user_id=b86d6ec3c196460c9c4e2fa4713fb2b7, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.183492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.197486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1260/2528] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:23 2022] PUT /identity/v3/projects/15dd70cd3158499996e666026e293811/users/b86d6ec3c196460c9c4e2fa4713fb2b7/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:23.202704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18094bdf-91f6-4260-9fb5-23d8b747950d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.202993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18094bdf-91f6-4260-9fb5-23d8b747950d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.203298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18094bdf-91f6-4260-9fb5-23d8b747950d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.223807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-18094bdf-91f6-4260-9fb5-23d8b747950d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:23.225696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18094bdf-91f6-4260-9fb5-23d8b747950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:23.286699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1269/2529] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:23.299576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.301381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:23.372030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:23.377119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7ccf9-33df-44c6-a6dd-ec290b762573 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.382668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.383485 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.383978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.384334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.394658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.396148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.414597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1261/2530] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:24.304939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451b343-4efe-496b-815b-e2d6fcc5a5b5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.305532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451b343-4efe-496b-815b-e2d6fcc5a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.305646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451b343-4efe-496b-815b-e2d6fcc5a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.326253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:24.337651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d451b343-4efe-496b-815b-e2d6fcc5a5b5 None None] MFA Rules not processed for user `b86d6ec3c196460c9c4e2fa4713fb2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:24.338876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d451b343-4efe-496b-815b-e2d6fcc5a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:24.416317 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1270/2531] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:24.432767 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:24.494821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c063df-36ab-4e5e-8e69-5ef1aafeed1f service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:24.498417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:24.499021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.499295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.499567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.509126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:24.510589 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:24.519971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1262/2532] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:25.919179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0d6b4-5885-483c-8321-9d729220e198 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:25.919641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0d6b4-5885-483c-8321-9d729220e198 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:25.919751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0d6b4-5885-483c-8321-9d729220e198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:25.939398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-70c0d6b4-5885-483c-8321-9d729220e198 None None] MFA Rules not processed for user `b86d6ec3c196460c9c4e2fa4713fb2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:25.940587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-70c0d6b4-5885-483c-8321-9d729220e198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:25.948826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1271/2533] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:26.038294 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:26.040007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:26.053610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91b699-01f8-4127-8c49-fe3711b22947 service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:26.059105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:26.059730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.060010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.060281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] PATH_INFO: `/v3/users/9fe75898e3b04ec0b1e17257654f7bc8` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.061400 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe75898e3b04ec0b1e17257654f7bc8)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:26.063347 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:26.115091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe75898e3b04ec0b1e17257654f7bc8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.124922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe75898e3b04ec0b1e17257654f7bc8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.125660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe75898e3b04ec0b1e17257654f7bc8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.133803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe75898e3b04ec0b1e17257654f7bc8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.146278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe75898e3b04ec0b1e17257654f7bc8'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.152446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1263/2534] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:26 2022] DELETE /identity/v3/users/9fe75898e3b04ec0b1e17257654f7bc8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:26.156932 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.157217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.157533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.177868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:26.179712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:26.240726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1272/2535] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:26.251129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:26.252590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:26.316916 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e24c9-85ba-490b-b7c2-f4cb46e17062 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:26.320987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:26.321636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.321922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.322199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.331782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:26.333244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:26.344075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1264/2536] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:26.683253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:26.685100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:26.710075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e173e24-cd9d-4814-8fec-7d9a7ca10de1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:26.716781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:26.717447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.717759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.718033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] PATH_INFO: `/v3/projects/fb4ba3c342a44899a0085770cf2f2aae` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.719157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4ba3c342a44899a0085770cf2f2aae)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:26.721381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:26.774477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4ba3c342a44899a0085770cf2f2aae', 'request_id': 'req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.781113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4ba3c342a44899a0085770cf2f2aae', 'request_id': 'req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.789703 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4ba3c342a44899a0085770cf2f2aae', 'request_id': 'req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.801012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] The token cache is being invalidate because project fb4ba3c342a44899a0085770cf2f2aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:26.801354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.802125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.805323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1273/2537] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:26 2022] DELETE /identity/v3/projects/fb4ba3c342a44899a0085770cf2f2aae => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:26.809256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:26.820864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9edc2c-3ae6-4fa0-bb73-dc5e3de1d428 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:26.825692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:26.896817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:26.897622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.898179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.898179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] PATH_INFO: `/v3/users/87aed76ac6174efd8e1d2ffb6087363d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.902161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87aed76ac6174efd8e1d2ffb6087363d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:26.904339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:26.951475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87aed76ac6174efd8e1d2ffb6087363d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.958406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87aed76ac6174efd8e1d2ffb6087363d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.959023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87aed76ac6174efd8e1d2ffb6087363d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.966237 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87aed76ac6174efd8e1d2ffb6087363d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.978853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87aed76ac6174efd8e1d2ffb6087363d'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.984662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1265/2538] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:26 2022] DELETE /identity/v3/users/87aed76ac6174efd8e1d2ffb6087363d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:27.263644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:27.265150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:27.280023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0ee23-7304-4fcf-9d3c-2cd5dec9fc9b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:27.294887 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:27.359067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:27.359717 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:27.360000 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:27.360276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] PATH_INFO: `/v3/projects/b6a44ccbf8274f74b971ff27523c9025` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:27.361412 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a44ccbf8274f74b971ff27523c9025)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:27.363363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:27.409642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a44ccbf8274f74b971ff27523c9025', 'request_id': 'req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.416007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a44ccbf8274f74b971ff27523c9025', 'request_id': 'req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.424086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a44ccbf8274f74b971ff27523c9025', 'request_id': 'req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.435056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] The token cache is being invalidate because project b6a44ccbf8274f74b971ff27523c9025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:27.435347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.435975 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.437422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1274/2539] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:27 2022] DELETE /identity/v3/projects/b6a44ccbf8274f74b971ff27523c9025 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:27.441540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:27.442907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:27.456376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005587a-b7c2-4ac5-9c6f-db7fd03cdeb4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:27.513394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:27.514062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:27.514348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:27.514628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] PATH_INFO: `/v3/users/958737f7fe3b4523ae7a94c0e28003ba` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:27.515747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=958737f7fe3b4523ae7a94c0e28003ba)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:27.517729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:27.550608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:27.566254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '958737f7fe3b4523ae7a94c0e28003ba'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.573423 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '958737f7fe3b4523ae7a94c0e28003ba'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.574084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '958737f7fe3b4523ae7a94c0e28003ba'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.581221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '958737f7fe3b4523ae7a94c0e28003ba'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.597819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-8da7aa97-510c-4b70-99ac-84e7a3c66200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '958737f7fe3b4523ae7a94c0e28003ba'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:27.605426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1266/2540] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:27 2022] DELETE /identity/v3/users/958737f7fe3b4523ae7a94c0e28003ba => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:27.934125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:27.935575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:27.955636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65e6e7-f1aa-46f7-abd7-c7e9677ebf7e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.014924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.015528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.015809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.016088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] PATH_INFO: `/v3/projects/48c18e61d9aa4dba95d33cdbba21b172` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.017395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c18e61d9aa4dba95d33cdbba21b172)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.019722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.064203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:28.065719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c18e61d9aa4dba95d33cdbba21b172', 'request_id': 'req-5af53eb8-5ae5-4900-b3e7-8f858204c57f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:28.074495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c18e61d9aa4dba95d33cdbba21b172', 'request_id': 'req-5af53eb8-5ae5-4900-b3e7-8f858204c57f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:28.081682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c18e61d9aa4dba95d33cdbba21b172', 'request_id': 'req-5af53eb8-5ae5-4900-b3e7-8f858204c57f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:28.093678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] The token cache is being invalidate because project 48c18e61d9aa4dba95d33cdbba21b172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:28.094075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:28.094700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5af53eb8-5ae5-4900-b3e7-8f858204c57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:28.096198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1275/2541] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:27 2022] DELETE /identity/v3/projects/48c18e61d9aa4dba95d33cdbba21b172 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:28.102865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.103151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.103428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.124844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:28.126113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:28.192418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1267/2542] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:28.202443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.202719 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.203004 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.222865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:28.224060 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:28.235813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1276/2543] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:28.240201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.251672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c71d7-b101-4ea6-8962-bcf74495feba None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.257752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.258405 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.258686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.258954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.259616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:28.259920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.261556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.275445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1268/2544] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:28.279949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.290647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57916d5-2361-4e98-b0b1-bb98431eb9fe None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.297144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.297848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.298073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.298528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.299341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.301541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.332974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1277/2545] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:28 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:28.338271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.348473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ea28f-2879-4d2f-bc45-d0aa9c00fc89 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.414355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.415949 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.416370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.416661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.417309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.419518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.449368 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1269/2546] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:28 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:28.451253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.462470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b529d3-991e-4650-8897-926216afd6f3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.467077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.467683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.467960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.468230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.468867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.470497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.480679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1278/2547] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:28.484642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.495762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2a2d4-6950-4ea3-8184-ad93f0bfab08 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.501387 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3507f-8836-473e-bfcf-9365d399b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.502026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3507f-8836-473e-bfcf-9365d399b854 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.502311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3507f-8836-473e-bfcf-9365d399b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.502572 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3507f-8836-473e-bfcf-9365d399b854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.503210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3507f-8836-473e-bfcf-9365d399b854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.504912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3507f-8836-473e-bfcf-9365d399b854 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.516887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1270/2548] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:28.521091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.532241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43300c51-40c8-4615-b1ed-b165f9e8bdfc admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.537220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.537925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.538209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.538747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] PATH_INFO: `/v3/projects/cca36573d59a4382956db081c1446697/users/6becbd7f997145d6a692e5837ab56070/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.553754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca36573d59a4382956db081c1446697, user_id=6becbd7f997145d6a692e5837ab56070, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.556545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.571345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1279/2549] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:28 2022] PUT /identity/v3/projects/cca36573d59a4382956db081c1446697/users/6becbd7f997145d6a692e5837ab56070/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:28.576379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7572a4a-0b0b-4841-b356-dfab85993f16 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.576724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7572a4a-0b0b-4841-b356-dfab85993f16 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.577008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7572a4a-0b0b-4841-b356-dfab85993f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.597602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-b7572a4a-0b0b-4841-b356-dfab85993f16 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:28.598825 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b7572a4a-0b0b-4841-b356-dfab85993f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:28.607747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:28.667550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1271/2550] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:28.674031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.675766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:28.748420 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2c05f-aada-4e1a-b695-e97210a0c3b8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.752256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.752960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.753316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.753694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.766605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.768256 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.783454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1280/2551] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:29.560960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.561534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.561641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.598055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] MFA Rules not processed for user `6becbd7f997145d6a692e5837ab56070`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:29.599356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:29.679834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1272/2552] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:29.684868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.698180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17ed04-618f-482b-b53a-2d5e00555ac1 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.703080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.703994 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.704280 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.704558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.705324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.707642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.747640 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1281/2553] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:29.749936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.762333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f243f-65d3-4371-99db-7ea953d0014e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.836594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.837228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.837538 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.837818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.838470 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.840468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.863609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1273/2554] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:29.867950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.879032 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf0f1a-35fd-4512-a695-79791215189b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.889404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.890093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.890323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.890603 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.891253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.892876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.915628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1282/2555] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:29.916792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.928501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf42839-1020-4882-91bc-a3aa6ccc5da0 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.933528 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.934201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.934510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.934790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] PATH_INFO: `/v3/projects/51fa4082ec39436e893a5050cf8e1c72/users/d2218f67eaac4a558f33883cea5e9bfd/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.951684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51fa4082ec39436e893a5050cf8e1c72, user_id=d2218f67eaac4a558f33883cea5e9bfd, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.955069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.970806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1274/2556] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:29 2022] PUT /identity/v3/projects/51fa4082ec39436e893a5050cf8e1c72/users/d2218f67eaac4a558f33883cea5e9bfd/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:29.975083 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.993130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bd867-71a8-44a3-af30-580a174b8fe1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:30.049291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:30.073998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd992e-c4cb-4cf0-9325-7cd736f49cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:30.074538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd992e-c4cb-4cf0-9325-7cd736f49cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.074845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd992e-c4cb-4cf0-9325-7cd736f49cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.075119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd992e-c4cb-4cf0-9325-7cd736f49cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.075760 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd992e-c4cb-4cf0-9325-7cd736f49cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:30.077354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd992e-c4cb-4cf0-9325-7cd736f49cd7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:30.091292 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1283/2557] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:30.092609 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:30.107080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1d81-30dd-4ae1-8196-8ccb2a404a2b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:30.112790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:30.113487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.113770 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.114048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] PATH_INFO: `/v3/projects/51fa4082ec39436e893a5050cf8e1c72/users/d2218f67eaac4a558f33883cea5e9bfd/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.116007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=51fa4082ec39436e893a5050cf8e1c72, user_id=d2218f67eaac4a558f33883cea5e9bfd, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:30.120454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:30.123963 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:30.137498 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1275/2558] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:30 2022] PUT /identity/v3/projects/51fa4082ec39436e893a5050cf8e1c72/users/d2218f67eaac4a558f33883cea5e9bfd/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:30.867316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.867827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.867827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.902614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] MFA Rules not processed for user `d2218f67eaac4a558f33883cea5e9bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:30.903635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:30.977764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1284/2559] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:30.989733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:30.991607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:31.052815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed201166-f6e0-470a-b327-518ad0b399cf admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.057283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.058116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.058542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.058822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.124577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.126120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.148898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1276/2560] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:32.514140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.515494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:32.528097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0793a-4034-4c1c-8be4-cb0cacc63afd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.531626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.532224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.532506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.532785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.541652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.543076 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.552466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1285/2561] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:39.724644 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.817040 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cd553-9379-46c7-9e49-ed40f0a35b9d service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.823185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.823943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.824226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.824503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.826369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.891715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:39.893161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:39.901932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:39.905565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1277/2562] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:42.548257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=50 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:43.522712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:43.524748 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.546609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad316-540e-4c63-9172-02dabe0d5b05 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.551304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.553061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.554244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.554688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] PATH_INFO: `/v3/users/16610ae2c3214396b78fb1caff01ab6e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.556731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16610ae2c3214396b78fb1caff01ab6e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.558266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:43.645860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16610ae2c3214396b78fb1caff01ab6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.656508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16610ae2c3214396b78fb1caff01ab6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.657335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16610ae2c3214396b78fb1caff01ab6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.658306 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16610ae2c3214396b78fb1caff01ab6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.669475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16610ae2c3214396b78fb1caff01ab6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.697538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1286/2563] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:43 2022] DELETE /identity/v3/users/16610ae2c3214396b78fb1caff01ab6e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:43.713069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:43.715109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.735667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d5c16-4424-4960-80a1-b7d30a9969ff service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.806459 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.806991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.807271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.807546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.809277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.908246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.909744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc57ed-4e92-4ba8-9d05-bcb86d353c9f service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:43.922011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:43.923659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1278/2564] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:43.924119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.956189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0beac3-4224-4c25-9f4b-ed13f2155108 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.962520 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.963127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.963350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.963803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] PATH_INFO: `/v3/projects/4fc4783cb8fe4ef6a78e4c541a408c9a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.964931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc4783cb8fe4ef6a78e4c541a408c9a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.966897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.075641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc4783cb8fe4ef6a78e4c541a408c9a', 'request_id': 'req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.083780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc4783cb8fe4ef6a78e4c541a408c9a', 'request_id': 'req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.094950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc4783cb8fe4ef6a78e4c541a408c9a', 'request_id': 'req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.107642 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] The token cache is being invalidate because project 4fc4783cb8fe4ef6a78e4c541a408c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:44.108195 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.108871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2a7f5069-a6fd-4f8e-86a8-1edd372de97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.110726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1287/2565] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:43 2022] DELETE /identity/v3/projects/4fc4783cb8fe4ef6a78e4c541a408c9a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:44.119865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.120650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.121041 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.146472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:44.146472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.225933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1279/2566] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:44.235025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.235025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.235025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.260027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:44.262376 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.276344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1288/2567] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:44.280334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.292254 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722f02-0001-4a66-9a42-e21e85dd7374 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.297520 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.298139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.298420 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.298691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.299338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:44.299641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.301284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.317319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1280/2568] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:44.322065 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.334372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b69c6-7113-4f55-8d0b-24d3286963c6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.339230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.339919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.340181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.340479 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.341113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.343043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.375559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1289/2569] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:44.377058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.387444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a9fe7-c86b-44d0-b683-9905f0817c11 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.424328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:44.450896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.451312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.451588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.452130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.452797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.454796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.481493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1281/2570] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:44.485638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.497370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36190b79-f433-49b1-802a-146b265feb87 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.502753 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.503386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.503672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.503947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.504601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.506275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.517217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1290/2571] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:44.522515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.532936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a086fff-0667-40e0-b2cd-73d928af4763 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.538215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e07a7-3efc-4c80-86a4-1f222d7afa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.538930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e07a7-3efc-4c80-86a4-1f222d7afa00 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.539358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e07a7-3efc-4c80-86a4-1f222d7afa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.539635 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e07a7-3efc-4c80-86a4-1f222d7afa00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.540277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e07a7-3efc-4c80-86a4-1f222d7afa00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.541908 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e07a7-3efc-4c80-86a4-1f222d7afa00 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.552964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1282/2572] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:44.557039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.568147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5582a-4dcd-40c9-ab74-b777f1bb7334 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.573263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.574049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.574411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.574836 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] PATH_INFO: `/v3/projects/be05c571ab2e49cbbdb2d4b7ffaeeded/users/5336ab72c1ad44e1adbdf762cd942539/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.590417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:44.593812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=be05c571ab2e49cbbdb2d4b7ffaeeded, user_id=5336ab72c1ad44e1adbdf762cd942539, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.598385 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.614362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1291/2573] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:44 2022] PUT /identity/v3/projects/be05c571ab2e49cbbdb2d4b7ffaeeded/users/5336ab72c1ad44e1adbdf762cd942539/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:44.627348 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.627670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.627961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.667569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] MFA Rules not processed for user `5336ab72c1ad44e1adbdf762cd942539`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:44.668836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.745300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1283/2574] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:44.757164 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.759940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.840819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85308bcc-985d-4c14-b01e-39d1e5f0d8cf admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.848759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.848759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.849099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.849982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.860244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.862402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.872067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1292/2575] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:47.099733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:47.101212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:47.192303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699dbf0-6da4-4ee1-bca3-ebdb0153a4de None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:47.198336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:47.198991 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:47.199281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:47.199559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] PATH_INFO: `/v3/users/6becbd7f997145d6a692e5837ab56070` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:47.200701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6becbd7f997145d6a692e5837ab56070)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:47.202698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:47.277852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6becbd7f997145d6a692e5837ab56070'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.279774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:47.287219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6becbd7f997145d6a692e5837ab56070'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.288549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6becbd7f997145d6a692e5837ab56070'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.301714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6becbd7f997145d6a692e5837ab56070'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.316799 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6becbd7f997145d6a692e5837ab56070'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.328140 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1284/2576] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:47 2022] DELETE /identity/v3/users/6becbd7f997145d6a692e5837ab56070 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:47.778492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:47.798389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0ee73-5d95-4a06-a0c4-4423838c49d5 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:47.890542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:47.891291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:47.891790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:47.892129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] PATH_INFO: `/v3/projects/cca36573d59a4382956db081c1446697` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:47.894641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca36573d59a4382956db081c1446697)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:47.897144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:47.987947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca36573d59a4382956db081c1446697', 'request_id': 'req-93ca0132-385e-470e-a01f-2414da7831a6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.999264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca36573d59a4382956db081c1446697', 'request_id': 'req-93ca0132-385e-470e-a01f-2414da7831a6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.010496 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca36573d59a4382956db081c1446697', 'request_id': 'req-93ca0132-385e-470e-a01f-2414da7831a6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.046846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] The token cache is being invalidate because project cca36573d59a4382956db081c1446697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:48.047149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.047845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.050089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1293/2577] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:47 2022] DELETE /identity/v3/projects/cca36573d59a4382956db081c1446697 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:48.054950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.056747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.074005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664038f-aa13-4485-aaf5-8f75fa632a47 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.126058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.127530 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.141411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:48.170324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.171007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.171262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.171537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] PATH_INFO: `/v3/users/d2218f67eaac4a558f33883cea5e9bfd` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.172846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2218f67eaac4a558f33883cea5e9bfd)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.174907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.214012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca0132-385e-470e-a01f-2414da7831a6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.218939 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.219599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.219930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.220178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] PATH_INFO: `/v3/users/a22c6bd0fac8443c844d7734aba97ef4` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.221621 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22c6bd0fac8443c844d7734aba97ef4)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.223811 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.228178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2218f67eaac4a558f33883cea5e9bfd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.236477 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2218f67eaac4a558f33883cea5e9bfd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.237215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2218f67eaac4a558f33883cea5e9bfd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.245025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2218f67eaac4a558f33883cea5e9bfd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.262356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2218f67eaac4a558f33883cea5e9bfd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.269515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1285/2578] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:48 2022] DELETE /identity/v3/users/d2218f67eaac4a558f33883cea5e9bfd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:48.286543 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22c6bd0fac8443c844d7734aba97ef4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.295982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22c6bd0fac8443c844d7734aba97ef4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.297117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22c6bd0fac8443c844d7734aba97ef4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.297812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22c6bd0fac8443c844d7734aba97ef4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.307510 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22c6bd0fac8443c844d7734aba97ef4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.333512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1294/2579] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:48 2022] DELETE /identity/v3/users/a22c6bd0fac8443c844d7734aba97ef4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:48.476275 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.478641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.528778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.530377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.559204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:48.581573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270037d-f032-4e41-b23c-f90c1f71c7e5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.607637 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad68c-fcef-4eae-8fbb-54a771c1aede admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.612413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.613244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.613576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.613922 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] PATH_INFO: `/v3/projects/c82c6a58da4641c18a387388d3b3ad73` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.615159 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82c6a58da4641c18a387388d3b3ad73)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.618017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.640641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.641282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.641597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.641875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] PATH_INFO: `/v3/projects/51fa4082ec39436e893a5050cf8e1c72` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.642986 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=51fa4082ec39436e893a5050cf8e1c72)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.645503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.650995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:48.668300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82c6a58da4641c18a387388d3b3ad73', 'request_id': 'req-14310469-a3a7-46ba-8783-d7614e8fcd27'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.675538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82c6a58da4641c18a387388d3b3ad73', 'request_id': 'req-14310469-a3a7-46ba-8783-d7614e8fcd27'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.685425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82c6a58da4641c18a387388d3b3ad73', 'request_id': 'req-14310469-a3a7-46ba-8783-d7614e8fcd27'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.696216 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] The token cache is being invalidate because project c82c6a58da4641c18a387388d3b3ad73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:48.696525 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.697166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-14310469-a3a7-46ba-8783-d7614e8fcd27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.698683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51fa4082ec39436e893a5050cf8e1c72', 'request_id': 'req-e05fd484-0795-4186-a004-de39c10375ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.698885 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1295/2580] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:48 2022] DELETE /identity/v3/projects/c82c6a58da4641c18a387388d3b3ad73 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:48.703526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a274176-03ed-4d07-888d-00a2311cdff3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.703810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a274176-03ed-4d07-888d-00a2311cdff3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.704087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a274176-03ed-4d07-888d-00a2311cdff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.705079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51fa4082ec39436e893a5050cf8e1c72', 'request_id': 'req-e05fd484-0795-4186-a004-de39c10375ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.712215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51fa4082ec39436e893a5050cf8e1c72', 'request_id': 'req-e05fd484-0795-4186-a004-de39c10375ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.724399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3a274176-03ed-4d07-888d-00a2311cdff3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:48.724785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] The token cache is being invalidate because project 51fa4082ec39436e893a5050cf8e1c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:48.725744 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.726161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3a274176-03ed-4d07-888d-00a2311cdff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.726476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e05fd484-0795-4186-a004-de39c10375ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:48.727981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1286/2581] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:48 2022] DELETE /identity/v3/projects/51fa4082ec39436e893a5050cf8e1c72 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:48.733805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad0050-493e-42da-83e6-d6b5a8eeb2e9 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.734166 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad0050-493e-42da-83e6-d6b5a8eeb2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.734509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad0050-493e-42da-83e6-d6b5a8eeb2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.755080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-45ad0050-493e-42da-83e6-d6b5a8eeb2e9 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:48.756895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-45ad0050-493e-42da-83e6-d6b5a8eeb2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.787843 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1296/2582] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:48.796345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.796635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.796916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.818247 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:48.819966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.826865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1287/2583] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:48.833630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.833921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.834202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.834686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1297/2584] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:48.838940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.848696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c91a6-b944-4776-8544-0dcc1a61d855 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.853943 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.854547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.854827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.855101 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.855766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:48.856070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.857782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.860441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:48.861720 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.869865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1298/2585] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:48.871654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1288/2586] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:48.875664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.876847 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.887465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722826b-4dad-49c3-8b99-e9d28ac479fd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.887659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe05257-50c5-4571-9961-fa8137f62e45 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.892181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.892319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.892786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.892919 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.897550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.897702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.897813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.897896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.897973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:48.898048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.898125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.898200 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.898312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.911974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:48.916021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1299/2587] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:48.918188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.929663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e9635-18b6-489e-a10a-2f51141952c4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:48.930638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1289/2588] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:48.934598 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.988721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:48.989255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:48.989575 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:48.989915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:48.991235 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:48.993508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:48.997522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc876816-6952-40e8-b43d-cf240fdf82b3 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.002209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.002818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.003105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.003382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.004010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.005983 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.023699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1300/2589] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:49.027974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.029298 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1290/2590] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:48 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:49.032953 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.038505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.043659 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdee66-6edb-4a8f-a51b-abe7ce1b2f6e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.093112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97006482-05a8-4518-816a-f48bd7e8e3ef admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.097834 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.098405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.098711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.098987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.099606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.101431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.102067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.102196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.102346 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.102622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.103257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.104853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.115056 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1291/2591] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:49.119454 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.130334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1301/2592] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:49.130575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243e268-497f-4d26-a5f3-b817de3dd46b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.131538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.135362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.135971 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.136257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.136539 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.137183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.138853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.141741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346275-bf16-4f17-9c46-397bb6b3c5e1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.148268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.148872 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.149151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.149421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.150148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.151853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.151959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1292/2593] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:49.155975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.165644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1302/2594] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:49.165802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dfd6c-3101-448d-980f-2d8cf3def3ce admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.166817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.170428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29b7d4-e16e-4167-957d-8d22dfa0b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.171080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29b7d4-e16e-4167-957d-8d22dfa0b9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.171371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29b7d4-e16e-4167-957d-8d22dfa0b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.171650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29b7d4-e16e-4167-957d-8d22dfa0b9a4 admin admin] PATH_INFO: `/v3/projects/d2df4e16a6944e35ba0c0693a91df64c/users/4617607cc218405d8097073a70f6f846/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.176947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb443ee-5104-4229-b858-3bfd34cc7f17 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.182248 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.182845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.183129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.183408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.184044 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.185700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.187061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29b7d4-e16e-4167-957d-8d22dfa0b9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2df4e16a6944e35ba0c0693a91df64c, user_id=4617607cc218405d8097073a70f6f846, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.189755 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29b7d4-e16e-4167-957d-8d22dfa0b9a4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.196254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1303/2595] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:49.200727 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.205417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1293/2596] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:49 2022] PUT /identity/v3/projects/d2df4e16a6944e35ba0c0693a91df64c/users/4617607cc218405d8097073a70f6f846/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:49.211346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c807b96-3525-4d5d-b63e-370ded287ee1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.215937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.216230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.216505 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.244987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] MFA Rules not processed for user `4617607cc218405d8097073a70f6f846`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:49.246182 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:49.272127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ade6-9dbd-4f96-988a-a1ff88b2f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.272897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ade6-9dbd-4f96-988a-a1ff88b2f84b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.273179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ade6-9dbd-4f96-988a-a1ff88b2f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.273454 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ade6-9dbd-4f96-988a-a1ff88b2f84b admin admin] PATH_INFO: `/v3/projects/4aaeb90daf154d50b53cdc4632968e9e/users/e956aa79f51f4be39a7006ba5cab5572/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.280162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.288442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2ade6-9dbd-4f96-988a-a1ff88b2f84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aaeb90daf154d50b53cdc4632968e9e, user_id=e956aa79f51f4be39a7006ba5cab5572, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.291149 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2ade6-9dbd-4f96-988a-a1ff88b2f84b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.304961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1304/2597] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:49 2022] PUT /identity/v3/projects/4aaeb90daf154d50b53cdc4632968e9e/users/e956aa79f51f4be39a7006ba5cab5572/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:49.314865 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.315042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1294/2598] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:49.315184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.315459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.319718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.330247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5b4d0-e544-4c0d-b173-107a552a28ef None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.346352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] MFA Rules not processed for user `e956aa79f51f4be39a7006ba5cab5572`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:49.347564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:49.399768 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.400492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.400782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.401059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.401745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.401888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.404238 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.422103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1305/2599] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:49.434192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.435651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:49.436356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1295/2600] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:49.440519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.510263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a04f-c735-4d86-822a-b16e1277c235 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.515566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.516225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.516510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.516788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.517445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.519763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.549030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1296/2601] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/users => generated 309 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:49.550632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.563288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a21ee-1d8d-4b4e-8959-b5f772d5864d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.571558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.572199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.572484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.572763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.573419 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.575084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.587501 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1297/2602] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:49.592881 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.605499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b976e15-14ed-4d53-a84c-91ccaa048d53 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.610011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.610672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.610956 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.611340 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] PATH_INFO: `/v3/projects/b1185a1d83c345219609d6469998632a/users/346ab53fc39b457ab5db144644c708e7/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.625228 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124e65b-16ee-4837-b76e-5fe93d2ce444 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.627086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1185a1d83c345219609d6469998632a, user_id=346ab53fc39b457ab5db144644c708e7, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.629765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.643057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1298/2603] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:49 2022] PUT /identity/v3/projects/b1185a1d83c345219609d6469998632a/users/346ab53fc39b457ab5db144644c708e7/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:49.647240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.660888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.684813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.685700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.686029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.686439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.718201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5dbc-bc26-4157-9869-0246cc4bd2bf admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.722882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.723490 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.723771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.724046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.724687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.726852 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.737518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1299/2604] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:49.742914 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.755405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.756913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.758513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509837b-2553-43f9-b4c6-770bf1f87df0 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.765809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2416c-8fdd-47f4-b3bb-7a0921fad977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.766829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2416c-8fdd-47f4-b3bb-7a0921fad977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.767285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2416c-8fdd-47f4-b3bb-7a0921fad977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.767425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1306/2605] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:49.768104 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2416c-8fdd-47f4-b3bb-7a0921fad977 admin admin] PATH_INFO: `/v3/projects/b1185a1d83c345219609d6469998632a/users/346ab53fc39b457ab5db144644c708e7/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.771469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2416c-8fdd-47f4-b3bb-7a0921fad977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1185a1d83c345219609d6469998632a, user_id=346ab53fc39b457ab5db144644c708e7, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.775210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2416c-8fdd-47f4-b3bb-7a0921fad977 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.791031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1300/2606] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:49 2022] PUT /identity/v3/projects/b1185a1d83c345219609d6469998632a/users/346ab53fc39b457ab5db144644c708e7/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:49.794548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.797307 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.797623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.797900 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.806937 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb271511-4863-4a69-96a3-edf71fa14d88 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.828201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] MFA Rules not processed for user `346ab53fc39b457ab5db144644c708e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:49.829492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:49.869678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.870347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.870630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.870905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] PATH_INFO: `/v3/users/e956aa79f51f4be39a7006ba5cab5572` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.872073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e956aa79f51f4be39a7006ba5cab5572)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.874012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.875790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.920221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1301/2607] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:49.931782 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.932858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e956aa79f51f4be39a7006ba5cab5572'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.940967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e956aa79f51f4be39a7006ba5cab5572'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.941758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e956aa79f51f4be39a7006ba5cab5572'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.942372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e956aa79f51f4be39a7006ba5cab5572'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.951737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6c1d5101-7b09-44c5-ba2e-5f30fe04e09c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e956aa79f51f4be39a7006ba5cab5572'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.974093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1307/2608] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:49 2022] DELETE /identity/v3/users/e956aa79f51f4be39a7006ba5cab5572 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:49.980298 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.981355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.981987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.993603 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7ab45-3e95-42fd-8712-1de15bb431de None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.003108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:50.004300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.029411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:50.046612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.047227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.047506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.047776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.049567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.066153 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1308/2609] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:50.080636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.082125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.147446 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86909e8-0d9d-484a-9775-724002a1840a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.151184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.151810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.152087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.152378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.162162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.163991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.174622 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1309/2610] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:50.247431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.248882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.259142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1302/2611] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:50.462035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.463765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.476700 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1219bb9-1a80-4246-94b8-ba70902e7884 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.482302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.485706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.486122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.486459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] PATH_INFO: `/v3/projects/4aaeb90daf154d50b53cdc4632968e9e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.487674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aaeb90daf154d50b53cdc4632968e9e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.489666 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.508840 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:50.537266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.538778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.541728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aaeb90daf154d50b53cdc4632968e9e', 'request_id': 'req-c3749203-bae3-49a2-b3ab-9dfecac14d8d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.548196 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aaeb90daf154d50b53cdc4632968e9e', 'request_id': 'req-c3749203-bae3-49a2-b3ab-9dfecac14d8d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.551807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df67024-74b9-49af-9e2f-20b53d78889b service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.555953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aaeb90daf154d50b53cdc4632968e9e', 'request_id': 'req-c3749203-bae3-49a2-b3ab-9dfecac14d8d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.568274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] The token cache is being invalidate because project 4aaeb90daf154d50b53cdc4632968e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:50.568654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.569809 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c3749203-bae3-49a2-b3ab-9dfecac14d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.571286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1310/2612] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:50 2022] DELETE /identity/v3/projects/4aaeb90daf154d50b53cdc4632968e9e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:50.576046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29cb4-c7d0-4b82-9229-37837cf3d149 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.576342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29cb4-c7d0-4b82-9229-37837cf3d149 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.576653 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29cb4-c7d0-4b82-9229-37837cf3d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.596709 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cee29cb4-c7d0-4b82-9229-37837cf3d149 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:50.597963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cee29cb4-c7d0-4b82-9229-37837cf3d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.612940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.613594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.613872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.614144 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] PATH_INFO: `/v3/users/4617607cc218405d8097073a70f6f846` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.615302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4617607cc218405d8097073a70f6f846)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.618457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.662853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1311/2613] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:50.665308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4617607cc218405d8097073a70f6f846'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.671512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.671796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.672064 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.674448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4617607cc218405d8097073a70f6f846'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.675360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4617607cc218405d8097073a70f6f846'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.684333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4617607cc218405d8097073a70f6f846'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.694274 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:50.695489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.697001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-d817d006-8dac-4d59-aecd-1c0dcfc9c81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4617607cc218405d8097073a70f6f846'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:50.703073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1303/2614] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:50 2022] DELETE /identity/v3/users/4617607cc218405d8097073a70f6f846 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:50.706170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1312/2615] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:50.707402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.707518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.707788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.708071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.716931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2f839-ca2e-4e06-8e28-21d8d73c2488 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.722363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:50.729838 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:50.731812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.774456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.775066 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.775352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.775627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.776268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:50.776571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.778193 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.789747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1313/2616] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:50.792883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1304/2617] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:50.794184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.802452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.804489 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e1b0f-b5a6-4bfb-851c-a63d2828d186 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.804969 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.809021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.809902 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.810188 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.810465 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.811108 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.812990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.841070 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1314/2618] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:50.845173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.858817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:50.863593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefb113-cd6e-4568-a6b5-b560dedec876 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.913523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01647b-9cea-41a7-810e-02a8e4dabf18 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.919377 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.919989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.920423 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.920702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.921326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.923768 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.926063 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.927201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.927357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.927637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.938934 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.940367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.947398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1315/2619] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:50.951521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.952557 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1305/2620] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:50.965121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca8d4e-2f01-453b-8c0a-040e63a87ecc admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.969866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.970469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.970747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.971020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.971682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.973254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.983878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1316/2621] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:50.988226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.998808 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df35c-45ea-45b7-b482-9a662d19f126 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.004684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23287a0-bd1a-47c6-b6e2-10f2c79b86de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.005098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23287a0-bd1a-47c6-b6e2-10f2c79b86de admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.005380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23287a0-bd1a-47c6-b6e2-10f2c79b86de admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.005712 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23287a0-bd1a-47c6-b6e2-10f2c79b86de admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.006880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23287a0-bd1a-47c6-b6e2-10f2c79b86de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.008995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23287a0-bd1a-47c6-b6e2-10f2c79b86de admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.023928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1306/2622] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:51.025120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.036611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29363fd-8acc-4654-bbbf-611b0fbd619f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.050114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.050928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.051508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.051959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] PATH_INFO: `/v3/projects/c8c0e351b22740849732bb995f4878b8/users/563b7749ad0d4417a2f73b178105dd8a/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.072142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c0e351b22740849732bb995f4878b8, user_id=563b7749ad0d4417a2f73b178105dd8a, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.074995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.090759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1317/2623] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:51 2022] PUT /identity/v3/projects/c8c0e351b22740849732bb995f4878b8/users/563b7749ad0d4417a2f73b178105dd8a/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:51.100988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.101356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.101856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.130667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] MFA Rules not processed for user `563b7749ad0d4417a2f73b178105dd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:51.132709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.185577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:51.211414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1307/2624] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:51.223023 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.224961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.273082 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.274593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.307921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e024d-44e5-4361-8b1e-764c5e97ea67 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.311802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.312437 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.312739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.313012 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.322286 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.323848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.335049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1318/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:51.346764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ed9f-f491-4595-81a9-b3ef94ea6460 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.355278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.355918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.356227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.356492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] PATH_INFO: `/v3/projects/d2df4e16a6944e35ba0c0693a91df64c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.357714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2df4e16a6944e35ba0c0693a91df64c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.359764 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.409267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2df4e16a6944e35ba0c0693a91df64c', 'request_id': 'req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.415807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2df4e16a6944e35ba0c0693a91df64c', 'request_id': 'req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.423691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2df4e16a6944e35ba0c0693a91df64c', 'request_id': 'req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.437928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] The token cache is being invalidate because project d2df4e16a6944e35ba0c0693a91df64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:51.438625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.439442 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.440862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1308/2626] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:51 2022] DELETE /identity/v3/projects/d2df4e16a6944e35ba0c0693a91df64c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:51.445359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.456469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9872b-22fa-4e9c-9472-47bc9d7b8a23 service glance] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.531816 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.532522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.532991 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.533390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] PATH_INFO: `/v3/users/346ab53fc39b457ab5db144644c708e7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.534799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=346ab53fc39b457ab5db144644c708e7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.536878 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.548711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:51.589107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '346ab53fc39b457ab5db144644c708e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.597338 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '346ab53fc39b457ab5db144644c708e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.598043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '346ab53fc39b457ab5db144644c708e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.598706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '346ab53fc39b457ab5db144644c708e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.607191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ac24f545-e58e-4fbb-b966-1136e6182dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '346ab53fc39b457ab5db144644c708e7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:51.626749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1319/2627] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:51 2022] DELETE /identity/v3/users/346ab53fc39b457ab5db144644c708e7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:51.919656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.921171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.936216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de36eb-5e72-41c9-9e23-6a38f8b00f2b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.994875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.995508 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.995791 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.996067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] PATH_INFO: `/v3/projects/b1185a1d83c345219609d6469998632a` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.997175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1185a1d83c345219609d6469998632a)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.999123 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.016036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:52.047910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1185a1d83c345219609d6469998632a', 'request_id': 'req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.054835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1185a1d83c345219609d6469998632a', 'request_id': 'req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.062452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1185a1d83c345219609d6469998632a', 'request_id': 'req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.074141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] The token cache is being invalidate because project b1185a1d83c345219609d6469998632a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:52.074438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.075083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f8c86a4a-5e37-4904-a90b-f6ab994f2fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.076578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1309/2628] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:51 2022] DELETE /identity/v3/projects/b1185a1d83c345219609d6469998632a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:52.081539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.081995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.082078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.105268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:52.106586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:52.177077 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1320/2629] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:52.186796 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.187073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.187357 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.214708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:52.216898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:52.226361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1310/2630] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:52.231297 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.245908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d819a-8749-4120-90c5-e72b35d79457 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.251121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.251817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.252114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.252431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.253099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:52.253483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.255169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.267988 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1321/2631] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:52.272948 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.284239 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831fc14-c283-4fb6-80e1-82a359eb7b6a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.289741 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.290350 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.290626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.290897 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.291542 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.293413 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.322012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1311/2632] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:52.326616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.336721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5928c2-1894-4577-b38b-161847c8fe16 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.394470 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.395081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.395355 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.395627 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.396253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.399433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.424890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1322/2633] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:52.429874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.440205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee6384-1e9d-47d2-b862-ac188e94c2e7 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.446818 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.447663 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.447786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.447966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.448608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.450261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.465513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1312/2634] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:52.469537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.476751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:52.480500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c4f03-245d-4669-bd98-738a8eee5883 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.486915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f72959-8d59-42cd-a154-5ec4aa2e2684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.487713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f72959-8d59-42cd-a154-5ec4aa2e2684 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.488163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f72959-8d59-42cd-a154-5ec4aa2e2684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.488440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f72959-8d59-42cd-a154-5ec4aa2e2684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.489081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f72959-8d59-42cd-a154-5ec4aa2e2684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.491378 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f72959-8d59-42cd-a154-5ec4aa2e2684 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.502819 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1323/2635] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:52.506954 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.516692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffc97c-dac9-4ab2-bcf9-8987f925ec1e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.521440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.522101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.522374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.522645 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] PATH_INFO: `/v3/projects/414f29248f784cb48b6602a8dbc6d176/users/14164bcc43d84ae985b5d9da79bec6c2/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.547282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414f29248f784cb48b6602a8dbc6d176, user_id=14164bcc43d84ae985b5d9da79bec6c2, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.549968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.564903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1313/2636] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:52 2022] PUT /identity/v3/projects/414f29248f784cb48b6602a8dbc6d176/users/14164bcc43d84ae985b5d9da79bec6c2/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:52.575720 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.575978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.576200 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.604506 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] MFA Rules not processed for user `14164bcc43d84ae985b5d9da79bec6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:52.605746 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:52.670178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1324/2637] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:52.675020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.742573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1460-e22a-4c7c-ba21-77993b3c2b28 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.747171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.747772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.748050 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.748320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.748935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.750846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.779207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1314/2638] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:52.783341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.794107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28de8d-3c74-4ed7-9edd-d17a2404ea83 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.853607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.854220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.854550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.854826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.855460 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.857388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.883056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1325/2639] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:52.887681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.909848 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02751e8d-bbd9-4db5-be42-6ff7e35a2cea admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.914625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.915227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.915506 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.915777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.916411 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.918038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.929249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1315/2640] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:52.933754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.945438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673e2f2-1680-4072-b4d5-5a5909d1aeb3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.950105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.950757 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.951036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.951311 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] PATH_INFO: `/v3/projects/4a2704827a5b487bb1b7ba1e35423cfb/users/56335a1c7328498daecccdbdd12f9884/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.966471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2704827a5b487bb1b7ba1e35423cfb, user_id=56335a1c7328498daecccdbdd12f9884, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.969115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.982691 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1326/2641] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:52 2022] PUT /identity/v3/projects/4a2704827a5b487bb1b7ba1e35423cfb/users/56335a1c7328498daecccdbdd12f9884/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:52.986739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.997484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227095a-0e43-47d2-957f-a57e035c9a27 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.009393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:53.058765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a462b-8a7e-4fad-ad25-37406941ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.059370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a462b-8a7e-4fad-ad25-37406941ea86 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.059648 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a462b-8a7e-4fad-ad25-37406941ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.059920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a462b-8a7e-4fad-ad25-37406941ea86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.060547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a462b-8a7e-4fad-ad25-37406941ea86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.062164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a462b-8a7e-4fad-ad25-37406941ea86 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.073122 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1316/2642] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:53.077382 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.088033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f7af-f0cf-4e06-92ad-fd2c3226868f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.092686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.093394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.093661 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.093940 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] PATH_INFO: `/v3/projects/4a2704827a5b487bb1b7ba1e35423cfb/users/56335a1c7328498daecccdbdd12f9884/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.095849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2704827a5b487bb1b7ba1e35423cfb, user_id=56335a1c7328498daecccdbdd12f9884, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.098522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.109113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1327/2643] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:53 2022] PUT /identity/v3/projects/4a2704827a5b487bb1b7ba1e35423cfb/users/56335a1c7328498daecccdbdd12f9884/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:53.120700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.120987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.121263 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.151511 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] MFA Rules not processed for user `56335a1c7328498daecccdbdd12f9884`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:53.152715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.220858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1317/2644] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:53.232882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.234591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.278047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:53.296295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148d01e-2ea0-4212-90c1-6ffb9f6ed659 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.300039 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.300689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.301103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.301404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.313079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.314751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.326570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1328/2645] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:53.422297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.432811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9348-ec18-4834-a95f-144405e0f276 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.443142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.443142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.448809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed75b1-9d06-46c4-902b-fc7c262b133c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.448809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed75b1-9d06-46c4-902b-fc7c262b133c service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.448809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed75b1-9d06-46c4-902b-fc7c262b133c service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.448809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed75b1-9d06-46c4-902b-fc7c262b133c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.512281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed75b1-9d06-46c4-902b-fc7c262b133c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.512974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05460cf-91da-4b0d-a7f2-9dd4d75fb069 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.513754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed75b1-9d06-46c4-902b-fc7c262b133c service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.517611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.518251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.518531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.519049 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] PATH_INFO: `/v3/users/5336ab72c1ad44e1adbdf762cd942539` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.520220 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5336ab72c1ad44e1adbdf762cd942539)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.523086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.526178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1318/2646] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:53.572464 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5336ab72c1ad44e1adbdf762cd942539'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.579641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5336ab72c1ad44e1adbdf762cd942539'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.580390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5336ab72c1ad44e1adbdf762cd942539'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.582795 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5336ab72c1ad44e1adbdf762cd942539'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.590080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5336ab72c1ad44e1adbdf762cd942539'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.610845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1329/2647] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:53 2022] DELETE /identity/v3/users/5336ab72c1ad44e1adbdf762cd942539 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:53.615874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.616159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.616426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.647170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:53.648686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.684610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:53.714105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1319/2648] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:53.724673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.727526 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.785205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946cb9d-ac8c-4d5f-a48b-cc16031b4922 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.789963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c511fa2-a9eb-44de-988d-5e1bdc84eff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.790567 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c511fa2-a9eb-44de-988d-5e1bdc84eff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.790844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c511fa2-a9eb-44de-988d-5e1bdc84eff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.791115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c511fa2-a9eb-44de-988d-5e1bdc84eff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.799905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c511fa2-a9eb-44de-988d-5e1bdc84eff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.801315 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c511fa2-a9eb-44de-988d-5e1bdc84eff4 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.806351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:53.817676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1330/2649] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:54.093882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.095324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:54.108811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42492151-d0b4-493a-a28d-4fd8125ad95e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.113500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.114145 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.114431 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.114705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] PATH_INFO: `/v3/projects/be05c571ab2e49cbbdb2d4b7ffaeeded` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.116788 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be05c571ab2e49cbbdb2d4b7ffaeeded)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.118774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.171299 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be05c571ab2e49cbbdb2d4b7ffaeeded', 'request_id': 'req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.177672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be05c571ab2e49cbbdb2d4b7ffaeeded', 'request_id': 'req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.186243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be05c571ab2e49cbbdb2d4b7ffaeeded', 'request_id': 'req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.197917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] The token cache is being invalidate because project be05c571ab2e49cbbdb2d4b7ffaeeded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:54.198261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.198894 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-04dd4b5b-2a6a-4e1f-9cfb-1c38a6d09268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.200373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1320/2650] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:54 2022] DELETE /identity/v3/projects/be05c571ab2e49cbbdb2d4b7ffaeeded => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:54.205237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.205758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.205866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.225550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:54.226764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:54.286255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1331/2651] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:54.295360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f0669-7843-433b-9c23-576f5c3986be None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.295650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f0669-7843-433b-9c23-576f5c3986be None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.295928 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f0669-7843-433b-9c23-576f5c3986be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.316730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-873f0669-7843-433b-9c23-576f5c3986be None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:54.317995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-873f0669-7843-433b-9c23-576f5c3986be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:54.326732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1321/2652] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:54.332419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.342095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3c09-d44d-4786-9de0-22a93047fe13 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.346635 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.347282 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.347554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.347824 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.348514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:54.348815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.350428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.362789 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1332/2653] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:54.369202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f0669-7843-433b-9c23-576f5c3986be None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.379044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f0669-7843-433b-9c23-576f5c3986be None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.384434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.385036 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.385360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.385665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.386297 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.388234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.418397 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1322/2654] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:54.419628 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.430080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34366ee-cb1d-4d5d-8ef7-40d7fbc73b5c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.487965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.488576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.488856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.489191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.489973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.492916 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.516604 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1333/2655] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:54.518147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.531016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f210-cd84-415a-8e83-71d9db6d6862 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.536018 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.536651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.536931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.538439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.539093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.540702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.551035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1323/2656] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:54.555336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.565236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa64e9-702b-40d5-8aa2-8496dfa32826 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.570531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbb852-46c3-4f3f-ad2f-51d9934f4694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.571162 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbb852-46c3-4f3f-ad2f-51d9934f4694 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.571439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbb852-46c3-4f3f-ad2f-51d9934f4694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.571855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbb852-46c3-4f3f-ad2f-51d9934f4694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.572534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbb852-46c3-4f3f-ad2f-51d9934f4694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.574166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbb852-46c3-4f3f-ad2f-51d9934f4694 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.585788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1334/2657] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:54.590171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.601059 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf843b-4ad1-430c-94df-699eb899599d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.605706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.606352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.606634 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.606911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] PATH_INFO: `/v3/projects/1cf381f6837348ec8bd7cc208afd725d/users/b903188d960144f8a87e0dec8ccdd205/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.622936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf381f6837348ec8bd7cc208afd725d, user_id=b903188d960144f8a87e0dec8ccdd205, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.625876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.642215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1324/2658] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:54 2022] PUT /identity/v3/projects/1cf381f6837348ec8bd7cc208afd725d/users/b903188d960144f8a87e0dec8ccdd205/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:54.655232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.655569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.655850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.686231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] MFA Rules not processed for user `b903188d960144f8a87e0dec8ccdd205`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:54.687431 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:54.770958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:54.786435 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1335/2659] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:54.787320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.857965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:54.863762 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33ffd6-e996-457e-b1e3-9ba6436930ee admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.877641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.878638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.879111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.885484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.885484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.885484 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.922895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1325/2660] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/projects => generated 393 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:54.928041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.943031 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ca9e1-63ef-4a8f-b3bf-64d2704104c2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.020966 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.022451 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.022740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.023181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.023820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.025965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.062007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1336/2661] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:54 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:55.066078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.083395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0ffa-070f-4ce9-896b-5d21fb513714 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.093105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.093775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.094054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.094330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.095714 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.097735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.120071 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1326/2662] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:55.123202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.136633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2941f76-1fdd-4d8e-9e21-d4b68dc78a74 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.143217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.144413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.144925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.145483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] PATH_INFO: `/v3/projects/87636e35bb354d9f848e9213f16aa183/users/d8c1e3a0b42f41deb6db1e471e7cf368/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.168478 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87636e35bb354d9f848e9213f16aa183, user_id=d8c1e3a0b42f41deb6db1e471e7cf368, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.174724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.202701 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1337/2663] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:55 2022] PUT /identity/v3/projects/87636e35bb354d9f848e9213f16aa183/users/d8c1e3a0b42f41deb6db1e471e7cf368/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:55.207045 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.234033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb656ece-c6d3-4958-a037-7fbf4245e6d4 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.296134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a8be-03ca-4ac3-b3a5-d2ad739a1d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.296751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a8be-03ca-4ac3-b3a5-d2ad739a1d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.297033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a8be-03ca-4ac3-b3a5-d2ad739a1d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.297312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a8be-03ca-4ac3-b3a5-d2ad739a1d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.297997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a8be-03ca-4ac3-b3a5-d2ad739a1d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.299576 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a8be-03ca-4ac3-b3a5-d2ad739a1d81 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.309550 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1327/2664] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:55.313716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.324475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1076b-ace7-4b74-aefc-c829a597e27f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.330029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.330690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.330978 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.331255 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] PATH_INFO: `/v3/projects/87636e35bb354d9f848e9213f16aa183/users/d8c1e3a0b42f41deb6db1e471e7cf368/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.333175 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=87636e35bb354d9f848e9213f16aa183, user_id=d8c1e3a0b42f41deb6db1e471e7cf368, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.336516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.347758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1338/2665] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:55 2022] PUT /identity/v3/projects/87636e35bb354d9f848e9213f16aa183/users/d8c1e3a0b42f41deb6db1e471e7cf368/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:55.358863 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.359150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.359429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.388227 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] MFA Rules not processed for user `d8c1e3a0b42f41deb6db1e471e7cf368`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:55.389430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.456839 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1328/2666] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:55.468580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.470330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.529167 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59707-c6f1-4530-bffa-37caa24a0fab admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.532764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.533362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.533678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.533948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.542996 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.544413 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.553652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1339/2667] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:55.604610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.614494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc95ab1-e921-4d30-a995-312e1ab1e461 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.618207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1a81c-8d3e-4e8b-9fcd-a69f87518681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.618803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1a81c-8d3e-4e8b-9fcd-a69f87518681 service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.619087 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1a81c-8d3e-4e8b-9fcd-a69f87518681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.619364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1a81c-8d3e-4e8b-9fcd-a69f87518681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.667178 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:55.678351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1a81c-8d3e-4e8b-9fcd-a69f87518681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.680076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1a81c-8d3e-4e8b-9fcd-a69f87518681 service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.690679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1329/2668] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:00.063723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.119063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:00.132910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5519f-9c81-4354-8028-4bc34b8427ba service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.137364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.137965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.138244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.138516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] PATH_INFO: `/v3/users/b903188d960144f8a87e0dec8ccdd205` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.139620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b903188d960144f8a87e0dec8ccdd205)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.141556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.191572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b903188d960144f8a87e0dec8ccdd205'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.199762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b903188d960144f8a87e0dec8ccdd205'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.200409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b903188d960144f8a87e0dec8ccdd205'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.201013 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b903188d960144f8a87e0dec8ccdd205'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.207974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b903188d960144f8a87e0dec8ccdd205'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.227047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1340/2669] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:00 2022] DELETE /identity/v3/users/b903188d960144f8a87e0dec8ccdd205 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:00.232246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.232738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.232857 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.252743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:00.254007 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.316626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1330/2670] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:00.326807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.328308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.389984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f9b7e-bbc2-44d0-9ded-087609fe97d9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.394147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.394769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.395058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.395340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.405384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.406856 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.416495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1341/2671] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:00.744069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.745989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.760362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ab6bc-d7be-46c0-9c72-189160156cae None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.766288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.767284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.767639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.768033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] PATH_INFO: `/v3/projects/1cf381f6837348ec8bd7cc208afd725d` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.769754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf381f6837348ec8bd7cc208afd725d)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.772054 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.826665 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf381f6837348ec8bd7cc208afd725d', 'request_id': 'req-1752a0be-099f-4d6f-9978-3cdb41d65c82'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.834655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf381f6837348ec8bd7cc208afd725d', 'request_id': 'req-1752a0be-099f-4d6f-9978-3cdb41d65c82'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.843654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf381f6837348ec8bd7cc208afd725d', 'request_id': 'req-1752a0be-099f-4d6f-9978-3cdb41d65c82'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.856024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] The token cache is being invalidate because project 1cf381f6837348ec8bd7cc208afd725d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:00.856391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.857065 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:00.858658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1331/2672] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:00 2022] DELETE /identity/v3/projects/1cf381f6837348ec8bd7cc208afd725d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:00.862754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.877854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d800f-c535-4176-a477-e9e3ce03839a service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.951181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.951877 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.952166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.952616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] PATH_INFO: `/v3/users/d8c1e3a0b42f41deb6db1e471e7cf368` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.953849 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c1e3a0b42f41deb6db1e471e7cf368)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.955951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.979330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:01.017005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c1e3a0b42f41deb6db1e471e7cf368'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.026556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c1e3a0b42f41deb6db1e471e7cf368'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.027268 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c1e3a0b42f41deb6db1e471e7cf368'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.027871 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c1e3a0b42f41deb6db1e471e7cf368'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.035974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-806f887e-14fe-452f-b095-f1b4c3048cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c1e3a0b42f41deb6db1e471e7cf368'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.057135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1342/2673] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:00 2022] DELETE /identity/v3/users/d8c1e3a0b42f41deb6db1e471e7cf368 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:01.348480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.350304 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:01.368678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752a0be-099f-4d6f-9978-3cdb41d65c82 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.433932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.434571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.434874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.435150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] PATH_INFO: `/v3/projects/87636e35bb354d9f848e9213f16aa183` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.436352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87636e35bb354d9f848e9213f16aa183)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.438282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.455024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:01.494575 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87636e35bb354d9f848e9213f16aa183', 'request_id': 'req-bd48edc6-a80f-45eb-8146-09efa6401e70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.502547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87636e35bb354d9f848e9213f16aa183', 'request_id': 'req-bd48edc6-a80f-45eb-8146-09efa6401e70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.513753 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87636e35bb354d9f848e9213f16aa183', 'request_id': 'req-bd48edc6-a80f-45eb-8146-09efa6401e70'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.525537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] The token cache is being invalidate because project 87636e35bb354d9f848e9213f16aa183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:01.525927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.526587 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-bd48edc6-a80f-45eb-8146-09efa6401e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:01.528451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1332/2674] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:01 2022] DELETE /identity/v3/projects/87636e35bb354d9f848e9213f16aa183 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:01.534794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.535971 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.536548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.565704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:01.566906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:01.626842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1343/2675] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:01.635398 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.635693 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.635987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.655787 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:01.656987 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:01.665417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1333/2676] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:01.669676 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.680227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdac3b-10e4-4fdb-a502-2bebed70542d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.685904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.686667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.687086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.687395 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.688552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:01.688965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.691109 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.704490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1344/2677] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:01.708920 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.718095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f7eb7-04e9-49ed-b00b-030d12f9ed26 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.722670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.723260 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.723541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.723820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.724985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.726944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.756250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1334/2678] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:01 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:01.760589 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.770862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a96b66-d4c7-4c57-a643-4d24471c64f6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.829208 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.829845 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.830123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.830391 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.831017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.832928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.852850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1345/2679] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:01 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:01.857296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.869455 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540fa4c-6953-48df-ad81-4017a7e20fdb admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.875507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.876164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.876449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.876724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.877371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.879011 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.896125 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1335/2680] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:01.900421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.910866 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6b8ea-7069-491d-af7c-11901c668b80 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.917069 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c1c60-c8af-4744-8c31-b9ce363cda39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.917749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c1c60-c8af-4744-8c31-b9ce363cda39 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.918063 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c1c60-c8af-4744-8c31-b9ce363cda39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.918313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c1c60-c8af-4744-8c31-b9ce363cda39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.919027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c1c60-c8af-4744-8c31-b9ce363cda39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.920829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c1c60-c8af-4744-8c31-b9ce363cda39 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.925253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:01.935176 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1346/2681] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:01.939418 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.949792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a04daa-7351-469f-a18f-edd972fd1d32 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.954375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.955035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.955317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.955605 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] PATH_INFO: `/v3/projects/d5f503c5441a4f69a0b55475bd859c5b/users/b986bbd4c70b4ceea08f116f37cb74f2/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.971090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f503c5441a4f69a0b55475bd859c5b, user_id=b986bbd4c70b4ceea08f116f37cb74f2, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.973910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.992739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1336/2682] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:01 2022] PUT /identity/v3/projects/d5f503c5441a4f69a0b55475bd859c5b/users/b986bbd4c70b4ceea08f116f37cb74f2/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:02.002859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.003142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.003417 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.033335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] MFA Rules not processed for user `b986bbd4c70b4ceea08f116f37cb74f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:02.034564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:02.101846 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1347/2683] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:02.114170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:02.116747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:02.174020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aedd9-cd6d-4a2f-bafd-f296f29d7273 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:02.178211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8b2a-fd1d-4455-a82c-319e3ede749a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:02.178835 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8b2a-fd1d-4455-a82c-319e3ede749a service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.179113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8b2a-fd1d-4455-a82c-319e3ede749a service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.179382 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8b2a-fd1d-4455-a82c-319e3ede749a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.189192 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8b2a-fd1d-4455-a82c-319e3ede749a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:02.190658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8b2a-fd1d-4455-a82c-319e3ede749a service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:02.199696 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1337/2684] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:02.222728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:02.294117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae76cc2-f083-44fd-b709-641508cc2223 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:02.298913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:02.299633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.300035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.300321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] PATH_INFO: `/v3/users/b986bbd4c70b4ceea08f116f37cb74f2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.301550 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b986bbd4c70b4ceea08f116f37cb74f2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:02.304503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:02.355804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b986bbd4c70b4ceea08f116f37cb74f2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.363380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b986bbd4c70b4ceea08f116f37cb74f2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.364033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b986bbd4c70b4ceea08f116f37cb74f2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.364684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b986bbd4c70b4ceea08f116f37cb74f2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.373538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b986bbd4c70b4ceea08f116f37cb74f2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.391495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1348/2685] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:02 2022] DELETE /identity/v3/users/b986bbd4c70b4ceea08f116f37cb74f2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:02.399171 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.399730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.399841 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.422425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:02.423864 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:02.466493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:02.495846 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1338/2686] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:02.505960 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:02.507345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:02.517266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:02.573686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a147d8-265c-46fd-b473-87544f1988fc admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:02.579683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dddd2e-36d7-491a-bb11-864cab91a830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:02.580741 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dddd2e-36d7-491a-bb11-864cab91a830 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.581265 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dddd2e-36d7-491a-bb11-864cab91a830 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.581810 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dddd2e-36d7-491a-bb11-864cab91a830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.593892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dddd2e-36d7-491a-bb11-864cab91a830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:02.595723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dddd2e-36d7-491a-bb11-864cab91a830 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:02.606817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1349/2687] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:02.905053 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:02.906546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:02.925343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecc9ee-c059-4ebc-b83d-cb454db742cc None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:02.930316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:02.931088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.931356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.931626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] PATH_INFO: `/v3/projects/d5f503c5441a4f69a0b55475bd859c5b` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.932851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f503c5441a4f69a0b55475bd859c5b)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:02.934834 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:02.984015 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f503c5441a4f69a0b55475bd859c5b', 'request_id': 'req-6ec37c68-1395-468a-b30c-8a46a39d1bec'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.991035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f503c5441a4f69a0b55475bd859c5b', 'request_id': 'req-6ec37c68-1395-468a-b30c-8a46a39d1bec'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:02.999691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f503c5441a4f69a0b55475bd859c5b', 'request_id': 'req-6ec37c68-1395-468a-b30c-8a46a39d1bec'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.010703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] The token cache is being invalidate because project d5f503c5441a4f69a0b55475bd859c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:03.011026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.011654 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6ec37c68-1395-468a-b30c-8a46a39d1bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.013136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1339/2688] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:02 2022] DELETE /identity/v3/projects/d5f503c5441a4f69a0b55475bd859c5b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:03.017968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.018271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.018534 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.038566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:03.039807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.100949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1350/2689] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:03.108828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.109113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.109394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.129705 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:03.131612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.139715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1340/2690] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:03.144387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.154969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fa3f8-f623-4cdc-b768-18aba672750d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.159615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.160230 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.160505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.160782 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.161592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:03.164275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.164275 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.177527 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1351/2691] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:03.183493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.194010 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5df0b-40b7-4bb5-8c56-ff7018d20ea8 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.198685 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.199312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.199588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.199856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.200497 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.202425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.232151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1341/2692] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:03.236405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.246360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a91ac-e5d5-4cd3-ab06-225c1dc91fa6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.306173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.306791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.307053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.307323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.307952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.309909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.332184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1352/2693] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:03.336671 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.346772 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad9a38-f00e-460f-9c66-358199ecad5c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.351596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.352186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.352461 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.352729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.353364 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.355331 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.365763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1342/2694] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:03.369486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.379925 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3792cbd-fcdc-40d7-8cc6-c0a21ab1c962 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.386078 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ef06b-380c-48c3-bc47-ea1d5e235001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.386688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ef06b-380c-48c3-bc47-ea1d5e235001 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.386969 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ef06b-380c-48c3-bc47-ea1d5e235001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.387245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ef06b-380c-48c3-bc47-ea1d5e235001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.387892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ef06b-380c-48c3-bc47-ea1d5e235001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.389522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ef06b-380c-48c3-bc47-ea1d5e235001 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.401532 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1353/2695] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:03.405700 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.415662 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46a01-4d39-4f8a-8a80-d8687faafed7 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.420146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.420790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.421066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.421337 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] PATH_INFO: `/v3/projects/a5901d59848f45d4bbbadbff624ea29a/users/6230252656a7459cad6b8fd419d35683/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.436021 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5901d59848f45d4bbbadbff624ea29a, user_id=6230252656a7459cad6b8fd419d35683, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.438680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.455316 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1343/2696] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:03 2022] PUT /identity/v3/projects/a5901d59848f45d4bbbadbff624ea29a/users/6230252656a7459cad6b8fd419d35683/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:03.465214 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.465533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.465815 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.493250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] MFA Rules not processed for user `6230252656a7459cad6b8fd419d35683`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:03.494426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.498374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:03.588038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1354/2697] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:03.598564 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.600376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.669708 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb6931-2f50-4b84-bba4-a3ffdd76e2d5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.673697 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409fd43-8642-4ce7-a1bb-1980f125350e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.674308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409fd43-8642-4ce7-a1bb-1980f125350e service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.674586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409fd43-8642-4ce7-a1bb-1980f125350e service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.674853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409fd43-8642-4ce7-a1bb-1980f125350e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.684701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409fd43-8642-4ce7-a1bb-1980f125350e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.686224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409fd43-8642-4ce7-a1bb-1980f125350e service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.689853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.691950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.696858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1344/2698] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:03.762574 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccff7f-a627-4029-8967-686725761d49 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:03.768888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:03.769587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.769868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.770141 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] PATH_INFO: `/v3/users/14164bcc43d84ae985b5d9da79bec6c2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.771952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=14164bcc43d84ae985b5d9da79bec6c2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:03.773913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:03.824785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14164bcc43d84ae985b5d9da79bec6c2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.831802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14164bcc43d84ae985b5d9da79bec6c2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.832514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14164bcc43d84ae985b5d9da79bec6c2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.833173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14164bcc43d84ae985b5d9da79bec6c2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.841008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14164bcc43d84ae985b5d9da79bec6c2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:03.861036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1355/2699] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:03 2022] DELETE /identity/v3/users/14164bcc43d84ae985b5d9da79bec6c2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:03.866776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.867199 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.867504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.876261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:03.889211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:03.891252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.990779 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1345/2700] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:03.997913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.999405 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:04.034381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:04.067443 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20a79c-f89c-4778-8f10-253af25d21cb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.071189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.071790 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.072068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.072337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.081231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.082876 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:04.096504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1356/2701] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:04.372110 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:04.373602 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:04.390467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a52a2a-327e-4e95-a34f-5836a3122103 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.395444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.396214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.396656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.396842 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] PATH_INFO: `/v3/projects/414f29248f784cb48b6602a8dbc6d176` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.398020 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414f29248f784cb48b6602a8dbc6d176)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.399959 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:04.447223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414f29248f784cb48b6602a8dbc6d176', 'request_id': 'req-08476919-f9f9-4d5b-bf02-3bb727d37788'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.453494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414f29248f784cb48b6602a8dbc6d176', 'request_id': 'req-08476919-f9f9-4d5b-bf02-3bb727d37788'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.460957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414f29248f784cb48b6602a8dbc6d176', 'request_id': 'req-08476919-f9f9-4d5b-bf02-3bb727d37788'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.471760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] The token cache is being invalidate because project 414f29248f784cb48b6602a8dbc6d176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:04.472052 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.472686 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.474781 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1346/2702] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:04 2022] DELETE /identity/v3/projects/414f29248f784cb48b6602a8dbc6d176 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:04.479081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:04.488602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61f28c-336b-4800-ac4a-565648d6bf0d service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.544688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.545243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.545556 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.545835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] PATH_INFO: `/v3/users/56335a1c7328498daecccdbdd12f9884` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.546964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=56335a1c7328498daecccdbdd12f9884)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.548884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:04.600683 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56335a1c7328498daecccdbdd12f9884'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.608363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56335a1c7328498daecccdbdd12f9884'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.609017 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56335a1c7328498daecccdbdd12f9884'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.609668 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56335a1c7328498daecccdbdd12f9884'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.618129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1edb8e35-36ea-4ac7-975c-6795ee4ab8bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56335a1c7328498daecccdbdd12f9884'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:04.635508 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1357/2703] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:04 2022] DELETE /identity/v3/users/56335a1c7328498daecccdbdd12f9884 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:04.913996 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:04.915468 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:04.929774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08476919-f9f9-4d5b-bf02-3bb727d37788 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.994719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.995355 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.995630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.995904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] PATH_INFO: `/v3/projects/4a2704827a5b487bb1b7ba1e35423cfb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.997002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2704827a5b487bb1b7ba1e35423cfb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.998930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.032339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:05.048776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2704827a5b487bb1b7ba1e35423cfb', 'request_id': 'req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.055274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2704827a5b487bb1b7ba1e35423cfb', 'request_id': 'req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.063241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2704827a5b487bb1b7ba1e35423cfb', 'request_id': 'req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.077252 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] The token cache is being invalidate because project 4a2704827a5b487bb1b7ba1e35423cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:05.077578 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.078215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-2a8c059d-d11c-4f45-8b1a-83b9f60d06ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.079868 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1347/2704] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:04 2022] DELETE /identity/v3/projects/4a2704827a5b487bb1b7ba1e35423cfb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:05.085232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.085607 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.085886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.106500 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:05.107693 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:05.148483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:05.168262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1358/2705] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:05.176136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.176518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.176877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.197927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:05.199121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:05.206929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1348/2706] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:05.211808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.221197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c90c7-cfcc-40d1-8a0e-2e8d86f3aa96 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.225751 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.226342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.226620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.226891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.227524 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:05.227848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.229433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.240704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1359/2707] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:05.244921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.254146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dffc9-3fc3-4a37-a475-b52274dd0333 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.259396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.259989 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.260274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.260552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.261226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.263311 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.291151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1349/2708] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:05.295371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.305726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccffe46-9fb1-49d2-91b5-554e85306f6b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.360611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.361213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.361569 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.361842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.362469 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.364372 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.385690 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1360/2709] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:05.389944 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.401250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdd229-32d6-4e0a-b961-13983fe78039 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.405880 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.406472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.406752 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.407026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.407657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.409247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.419077 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1350/2710] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:05.422891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.432652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5340ab-b3e2-4dd6-bbf7-8819bbafb763 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.437140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f997e-aa8b-46ae-a3c5-89511af4456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.437756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f997e-aa8b-46ae-a3c5-89511af4456a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.438036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f997e-aa8b-46ae-a3c5-89511af4456a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.438308 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f997e-aa8b-46ae-a3c5-89511af4456a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.438930 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f997e-aa8b-46ae-a3c5-89511af4456a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.440497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f997e-aa8b-46ae-a3c5-89511af4456a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.452369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1361/2711] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:05.456661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.467051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a75f3-c7da-4dc7-97b7-e866fadc88b1 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.471642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.472283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.472568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.472849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] PATH_INFO: `/v3/projects/b81b77a0fce84e6e90bc18ccbc271f31/users/3621616211bb4dc5b1722c5c511fad85/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.487775 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81b77a0fce84e6e90bc18ccbc271f31, user_id=3621616211bb4dc5b1722c5c511fad85, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.491070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.503358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1351/2712] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:05 2022] PUT /identity/v3/projects/b81b77a0fce84e6e90bc18ccbc271f31/users/3621616211bb4dc5b1722c5c511fad85/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:05.514548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.514854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.515110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.544859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] MFA Rules not processed for user `3621616211bb4dc5b1722c5c511fad85`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:05.546125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:05.609212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1362/2713] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:05.613731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.680215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48754363-ed77-4a13-b285-c0956c6ef32d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.685049 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.685723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.686005 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.686277 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.686915 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.688792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.720328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1352/2714] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:05.725326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.735904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df453f16-5dd3-43c4-84b7-4d819c8a5178 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.776323 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:05.793879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.794491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.794770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.795038 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.795652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.797571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.818264 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1363/2715] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:05.822175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.832191 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe825e-ca47-4321-b1ae-44eec32485da admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.837428 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.838048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.838327 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.838596 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.839224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.840820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.854194 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1353/2716] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:05.858450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.869674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26743ccc-2a66-491a-a931-414f2178e8c7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.874331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.875035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.875316 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.875590 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] PATH_INFO: `/v3/projects/45f2eefafaff4bd9997068e5ec0d57a3/users/e1b635222ec34f6a99c33e9a0e9adacd/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.898781 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f2eefafaff4bd9997068e5ec0d57a3, user_id=e1b635222ec34f6a99c33e9a0e9adacd, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.901347 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.915048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1364/2717] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:05 2022] PUT /identity/v3/projects/45f2eefafaff4bd9997068e5ec0d57a3/users/e1b635222ec34f6a99c33e9a0e9adacd/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:05.919223 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.929433 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b5d6f-51d9-496a-8573-6b42d4e49555 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.959090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:05.996197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eca79a-b112-42e1-a193-562193850f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.996811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eca79a-b112-42e1-a193-562193850f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.997093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eca79a-b112-42e1-a193-562193850f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.997370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eca79a-b112-42e1-a193-562193850f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.998037 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eca79a-b112-42e1-a193-562193850f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.999657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eca79a-b112-42e1-a193-562193850f36 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.010026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1354/2718] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:06.014415 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:06.027337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a884292-c43e-43ec-a74f-ee959670e9e3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.032097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.032854 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.033135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.033407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] PATH_INFO: `/v3/projects/45f2eefafaff4bd9997068e5ec0d57a3/users/e1b635222ec34f6a99c33e9a0e9adacd/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.035319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f2eefafaff4bd9997068e5ec0d57a3, user_id=e1b635222ec34f6a99c33e9a0e9adacd, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.038090 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.048972 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1365/2719] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:06 2022] PUT /identity/v3/projects/45f2eefafaff4bd9997068e5ec0d57a3/users/e1b635222ec34f6a99c33e9a0e9adacd/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:06.061940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.062261 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.062502 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.098080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] MFA Rules not processed for user `e1b635222ec34f6a99c33e9a0e9adacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:06.099317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.188320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1355/2720] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:06.200294 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:06.202226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.260344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61615731-6c4b-4282-902b-2fd99db53dc0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.263920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8b56b-17e9-487f-b749-1fb74bf489b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.264521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b56b-17e9-487f-b749-1fb74bf489b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.264793 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b56b-17e9-487f-b749-1fb74bf489b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.265061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b56b-17e9-487f-b749-1fb74bf489b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.283818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8b56b-17e9-487f-b749-1fb74bf489b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.285318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8b56b-17e9-487f-b749-1fb74bf489b1 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.295466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1366/2721] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:06.363743 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:06.436913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100ab82-e51b-43b9-9d9c-c71e4ae8f8ac None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.442284 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.442985 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.443268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.443541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] PATH_INFO: `/v3/users/3621616211bb4dc5b1722c5c511fad85` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.444691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3621616211bb4dc5b1722c5c511fad85)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.446672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.496995 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3621616211bb4dc5b1722c5c511fad85'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:06.500938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:06.505173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3621616211bb4dc5b1722c5c511fad85'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:06.506531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3621616211bb4dc5b1722c5c511fad85'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:06.513984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3621616211bb4dc5b1722c5c511fad85'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:06.526921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3621616211bb4dc5b1722c5c511fad85'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:06.532899 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1356/2722] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:06 2022] DELETE /identity/v3/users/3621616211bb4dc5b1722c5c511fad85 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:06.538005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.538295 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.538597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.557421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:06.558655 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.621429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1367/2723] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:06.631926 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:06.633314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.704430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da458f-69cb-4868-b227-7fdb20f5a224 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.711078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.711718 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.712002 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.712324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.723656 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.725302 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.738241 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1357/2724] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:07.024838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:07.026538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:07.039864 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0505b-abb9-4231-a68a-30c253a79be7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:07.045146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:07.045909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:07.046253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:07.046740 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] PATH_INFO: `/v3/projects/b81b77a0fce84e6e90bc18ccbc271f31` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:07.048113 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81b77a0fce84e6e90bc18ccbc271f31)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:07.049787 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:07.092313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:07.108438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81b77a0fce84e6e90bc18ccbc271f31', 'request_id': 'req-6244410f-50c6-4e53-a472-9c5bfe27658d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.117387 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81b77a0fce84e6e90bc18ccbc271f31', 'request_id': 'req-6244410f-50c6-4e53-a472-9c5bfe27658d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.127257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81b77a0fce84e6e90bc18ccbc271f31', 'request_id': 'req-6244410f-50c6-4e53-a472-9c5bfe27658d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.139303 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] The token cache is being invalidate because project b81b77a0fce84e6e90bc18ccbc271f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:07.139595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.140234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.141762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1368/2725] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:07 2022] DELETE /identity/v3/projects/b81b77a0fce84e6e90bc18ccbc271f31 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:07.146186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:07.165480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8eec4-233a-4351-a56c-57709501704b service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:07.267240 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:07.267873 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:07.268150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:07.268424 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] PATH_INFO: `/v3/users/e1b635222ec34f6a99c33e9a0e9adacd` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:07.269819 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b635222ec34f6a99c33e9a0e9adacd)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:07.271771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:07.323722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b635222ec34f6a99c33e9a0e9adacd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.333022 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b635222ec34f6a99c33e9a0e9adacd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.333978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b635222ec34f6a99c33e9a0e9adacd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.348849 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b635222ec34f6a99c33e9a0e9adacd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.358351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:07.364118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f20d75aa-6312-4d3a-bc29-1c2499ee7cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b635222ec34f6a99c33e9a0e9adacd'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.373073 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1358/2726] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:07 2022] DELETE /identity/v3/users/e1b635222ec34f6a99c33e9a0e9adacd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:07.946514 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:07.948009 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:07.963523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244410f-50c6-4e53-a472-9c5bfe27658d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.021197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.021861 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.022138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.022411 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] PATH_INFO: `/v3/projects/45f2eefafaff4bd9997068e5ec0d57a3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.023504 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f2eefafaff4bd9997068e5ec0d57a3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.025388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.072551 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f2eefafaff4bd9997068e5ec0d57a3', 'request_id': 'req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.078823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f2eefafaff4bd9997068e5ec0d57a3', 'request_id': 'req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.086671 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f2eefafaff4bd9997068e5ec0d57a3', 'request_id': 'req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.097728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] The token cache is being invalidate because project 45f2eefafaff4bd9997068e5ec0d57a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:08.098015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.098633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ba9d46b-7ae4-45d8-9651-e1bb4cd1fe7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.100321 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1369/2727] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:07 2022] DELETE /identity/v3/projects/45f2eefafaff4bd9997068e5ec0d57a3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:08.105169 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.106162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.106162 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.125943 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:08.127136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:08.187142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1359/2728] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:08.194882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.195166 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.195441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.215633 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:08.216745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:08.225147 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1370/2729] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:08.229248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.238910 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6a259-52ee-4c29-aab5-08a8d1de543b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.244359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.244966 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.245251 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.245565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.246229 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:08.246537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.248181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.263682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1360/2730] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:08.264813 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.274088 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c95b8-3932-4738-8345-2273a498c5d6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.278570 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.279157 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.279546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.279823 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.280438 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.282348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.309205 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1371/2731] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:08.313573 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.323481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8df719-62b9-4fac-8101-c80d90942153 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.382785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.383404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.383691 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.383978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.384625 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.386591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.407384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1361/2732] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:08.412459 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.422593 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfa434-664d-4caf-a246-4862d8bfd304 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.427747 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.428334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.428611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.428882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.429548 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.431119 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.441245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1372/2733] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:08.446279 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.456172 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1d74-82e3-4948-96d9-ad2565629b21 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.460997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc080f6-9bb3-4573-9217-aabe9adae441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.461627 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc080f6-9bb3-4573-9217-aabe9adae441 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.461913 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc080f6-9bb3-4573-9217-aabe9adae441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.462189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc080f6-9bb3-4573-9217-aabe9adae441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.462829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc080f6-9bb3-4573-9217-aabe9adae441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.464429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc080f6-9bb3-4573-9217-aabe9adae441 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.476672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1362/2734] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:08.480657 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.491337 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a40d8-99dd-4ccb-8631-8566c63cc96b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.495800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.496440 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.496721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.496995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] PATH_INFO: `/v3/projects/e4b4d044b9314b008524e8b656bd25da/users/ea28d97069954d669662aeb2c7497931/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.504833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:08.512705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b4d044b9314b008524e8b656bd25da, user_id=ea28d97069954d669662aeb2c7497931, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.515848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.529025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1373/2735] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:08 2022] PUT /identity/v3/projects/e4b4d044b9314b008524e8b656bd25da/users/ea28d97069954d669662aeb2c7497931/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:08.539807 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.540094 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.540367 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.567998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] MFA Rules not processed for user `ea28d97069954d669662aeb2c7497931`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:08.569218 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:08.614096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:08.635393 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1363/2736] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:08.639927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.701835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efed04d-d587-41df-a451-7f07a2cbfd2d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.706389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.707015 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.707288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.707554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.708222 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.710716 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.736409 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1374/2737] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:08.740469 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.752023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177f5c6-14f0-4294-bbfd-ff64ada088cc None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.807682 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.808362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.808642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.808912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.809558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.811444 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.836896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1364/2738] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:08 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:08.838121 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.848571 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef22326-e3da-4ff5-915d-3077dba98be2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.854202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.854838 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.855117 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.855418 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.856071 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.857718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.868486 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1375/2739] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:08.872629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.883376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f4db1-187e-4315-ac81-bdc6721c14c5 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.887994 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.888650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.888929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.889201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] PATH_INFO: `/v3/projects/23c6fa4d10274a2c81bc3b6b4702ca8e/users/548931d8ad8a4ab4aad2332f357bb5c7/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.905754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c6fa4d10274a2c81bc3b6b4702ca8e, user_id=548931d8ad8a4ab4aad2332f357bb5c7, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.908437 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.921369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1365/2740] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:08 2022] PUT /identity/v3/projects/23c6fa4d10274a2c81bc3b6b4702ca8e/users/548931d8ad8a4ab4aad2332f357bb5c7/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:08.925324 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:08.935636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca4402-6869-45a3-9d48-6c94f4072c1c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.994241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf7228-710d-43ae-bd4d-87e3d75f7fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.994853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf7228-710d-43ae-bd4d-87e3d75f7fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.995129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf7228-710d-43ae-bd4d-87e3d75f7fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.995401 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf7228-710d-43ae-bd4d-87e3d75f7fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.996026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf7228-710d-43ae-bd4d-87e3d75f7fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.997662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf7228-710d-43ae-bd4d-87e3d75f7fd2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:09.007879 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1376/2741] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:09.012142 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:09.022657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68df62-917a-4356-bc87-cd8b31acb5f8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:09.027226 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:09.027872 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:09.028150 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:09.028426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] PATH_INFO: `/v3/projects/23c6fa4d10274a2c81bc3b6b4702ca8e/users/548931d8ad8a4ab4aad2332f357bb5c7/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:09.030358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c6fa4d10274a2c81bc3b6b4702ca8e, user_id=548931d8ad8a4ab4aad2332f357bb5c7, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:09.032988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:09.043809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1366/2742] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:09 2022] PUT /identity/v3/projects/23c6fa4d10274a2c81bc3b6b4702ca8e/users/548931d8ad8a4ab4aad2332f357bb5c7/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:09.055609 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:09.055905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:09.056185 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:09.084800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] MFA Rules not processed for user `548931d8ad8a4ab4aad2332f357bb5c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:09.086036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:09.129949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:09.155094 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1377/2743] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:09.166493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:09.168212 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:09.225268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31af78a-f6c6-464d-a8b0-b52f2c7c5b63 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:09.228912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763402-2582-4623-8044-d000776afa4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:09.229544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763402-2582-4623-8044-d000776afa4b service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:09.229829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763402-2582-4623-8044-d000776afa4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:09.230107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763402-2582-4623-8044-d000776afa4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:09.238975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50763402-2582-4623-8044-d000776afa4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:09.241025 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50763402-2582-4623-8044-d000776afa4b service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:09.250440 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1367/2744] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:09.787673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:09.848595 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad11dc-e475-4ff5-9828-7387a8398dab None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:09.853300 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:09.853958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:09.854237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:09.854507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] PATH_INFO: `/v3/users/ea28d97069954d669662aeb2c7497931` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:09.855600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea28d97069954d669662aeb2c7497931)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:09.857576 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:09.909654 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea28d97069954d669662aeb2c7497931'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:09.917853 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea28d97069954d669662aeb2c7497931'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:09.918505 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea28d97069954d669662aeb2c7497931'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:09.919112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea28d97069954d669662aeb2c7497931'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:09.926047 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea28d97069954d669662aeb2c7497931'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:09.943998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1378/2745] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:09 2022] DELETE /identity/v3/users/ea28d97069954d669662aeb2c7497931 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:09.948523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:09.948810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:09.949088 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:09.968081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:09.969274 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.007822 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:10.030975 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1368/2746] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:10.040818 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.042206 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.098999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0cd0-082a-48e5-9628-270ac6c3e722 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.102625 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.103227 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.103509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.103784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.112732 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.114184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.124099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1379/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:10.400978 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.403728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.417161 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb5381-5835-4482-b956-fcb23262b8a3 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.422273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.422936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.423403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.424351 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] PATH_INFO: `/v3/projects/e4b4d044b9314b008524e8b656bd25da` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.426032 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b4d044b9314b008524e8b656bd25da)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.429107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.488152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b4d044b9314b008524e8b656bd25da', 'request_id': 'req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.496467 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b4d044b9314b008524e8b656bd25da', 'request_id': 'req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.506722 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b4d044b9314b008524e8b656bd25da', 'request_id': 'req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.517776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] The token cache is being invalidate because project e4b4d044b9314b008524e8b656bd25da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:10.518070 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.519090 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.522246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1369/2748] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:10 2022] DELETE /identity/v3/projects/e4b4d044b9314b008524e8b656bd25da => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:10.526448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.532539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:10.537737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcea8-f4cc-4e4a-a334-8038155599d2 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.601198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.602099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.602626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.603068 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] PATH_INFO: `/v3/users/548931d8ad8a4ab4aad2332f357bb5c7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.604341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548931d8ad8a4ab4aad2332f357bb5c7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.606429 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.666775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548931d8ad8a4ab4aad2332f357bb5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.677895 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548931d8ad8a4ab4aad2332f357bb5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.678615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548931d8ad8a4ab4aad2332f357bb5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.679254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548931d8ad8a4ab4aad2332f357bb5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.692035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548931d8ad8a4ab4aad2332f357bb5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.711129 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1380/2749] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:10 2022] DELETE /identity/v3/users/548931d8ad8a4ab4aad2332f357bb5c7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:11.038167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.040121 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.057282 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9d6d8-d757-4032-ae7e-c0e3d7e461ab admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.086949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.089043 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.127895 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.128571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.128853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.129124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] PATH_INFO: `/v3/projects/23c6fa4d10274a2c81bc3b6b4702ca8e` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.130930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c6fa4d10274a2c81bc3b6b4702ca8e)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.133314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.171848 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95930056-a94d-4566-9cb1-0e1a9a54a127 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.176477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.177860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.178189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.178402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] PATH_INFO: `/v3/users/6230252656a7459cad6b8fd419d35683` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.179554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6230252656a7459cad6b8fd419d35683)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.181480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.187570 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c6fa4d10274a2c81bc3b6b4702ca8e', 'request_id': 'req-14b16d4c-51c4-4409-90c8-50582e5b9d51'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.194363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c6fa4d10274a2c81bc3b6b4702ca8e', 'request_id': 'req-14b16d4c-51c4-4409-90c8-50582e5b9d51'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.202236 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c6fa4d10274a2c81bc3b6b4702ca8e', 'request_id': 'req-14b16d4c-51c4-4409-90c8-50582e5b9d51'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.208249 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:11.212327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:11.218408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] The token cache is being invalidate because project 23c6fa4d10274a2c81bc3b6b4702ca8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:11.218877 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.219802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-14b16d4c-51c4-4409-90c8-50582e5b9d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.222813 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1370/2750] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:11 2022] DELETE /identity/v3/projects/23c6fa4d10274a2c81bc3b6b4702ca8e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:11.228509 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42affd-865b-444c-997b-2b6d9dbd8258 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.228798 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42affd-865b-444c-997b-2b6d9dbd8258 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.229086 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42affd-865b-444c-997b-2b6d9dbd8258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.237155 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6230252656a7459cad6b8fd419d35683'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.245442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6230252656a7459cad6b8fd419d35683'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.246158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6230252656a7459cad6b8fd419d35683'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.246803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6230252656a7459cad6b8fd419d35683'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.250518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ab42affd-865b-444c-997b-2b6d9dbd8258 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:11.251747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ab42affd-865b-444c-997b-2b6d9dbd8258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.254290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-449a2944-2332-422b-94e5-28ea953e9742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6230252656a7459cad6b8fd419d35683'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.273102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1381/2751] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:11 2022] DELETE /identity/v3/users/6230252656a7459cad6b8fd419d35683 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:11.277913 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.278194 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.278612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.298502 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:11.299695 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.316647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1371/2752] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:11.324374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.324599 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.324979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.346187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:11.347390 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.371312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1382/2753] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:11.382542 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.383974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.420519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1372/2754] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:11.424391 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.434434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c05df-c4fb-46c8-ba4d-8e290051de83 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.439647 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.440600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.440907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.441187 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.441907 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:11.442211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.444255 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.456631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b88bd-4808-49a8-95fd-1522b6ed29a6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.458725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1373/2755] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:07:11.461430 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.462224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.462562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.462842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.463568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.472903 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.474709 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1129d-5c35-4238-827e-36003233a538 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.475266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.479678 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.480280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.480608 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.480886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.481546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.483529 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.501046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1383/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:11.525905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1374/2757] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:11.529989 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.541775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf34ca-cd9a-481c-a73c-8ccd98ad4b6c service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.599766 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:11.615867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.616515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.616767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.617079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.617804 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.619755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.641232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1384/2758] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:11.646283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.658228 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda6444-d284-4bad-94e9-d79c0fc788b8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.664016 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.664655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.664932 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.665205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.665882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.667510 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.679141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1375/2759] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:11.683290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.695731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7301d87-0638-4fc9-9052-6737b65909be admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.700224 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf4da9-a290-4097-a417-995fb9aa88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.700820 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4da9-a290-4097-a417-995fb9aa88b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.701096 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4da9-a290-4097-a417-995fb9aa88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.701369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4da9-a290-4097-a417-995fb9aa88b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.702036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf4da9-a290-4097-a417-995fb9aa88b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.703629 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf4da9-a290-4097-a417-995fb9aa88b0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.714517 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1385/2760] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:11.718694 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.729486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471897a-b650-4b43-a303-745e3fabb4db admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.734336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.734998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.735288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.735567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] PATH_INFO: `/v3/projects/fd3cd3613caa4eaa9a7ea3cc7bf3333a/users/33917147fa3a430c82475ed7ca836cfb/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.751225 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3cd3613caa4eaa9a7ea3cc7bf3333a, user_id=33917147fa3a430c82475ed7ca836cfb, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.754265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.769552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1376/2761] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:11 2022] PUT /identity/v3/projects/fd3cd3613caa4eaa9a7ea3cc7bf3333a/users/33917147fa3a430c82475ed7ca836cfb/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:11.781107 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.781392 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.781705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.811345 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] MFA Rules not processed for user `33917147fa3a430c82475ed7ca836cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:11.812599 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.823681 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.826128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.860084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:11.898984 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470eb0a7-3b12-4478-b29d-e664b6f3dd6e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.903815 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.904398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1386/2762] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:11.908012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.908305 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.908544 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.908661 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] PATH_INFO: `/v3/projects/a5901d59848f45d4bbbadbff624ea29a` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.909800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5901d59848f45d4bbbadbff624ea29a)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.912410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.919394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7079a-91b1-4d51-9adf-ce09df00af57 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.923979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.924568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.924842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.925112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.925786 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.928350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.957348 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1387/2763] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:11.961918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.963651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5901d59848f45d4bbbadbff624ea29a', 'request_id': 'req-9a292532-f032-45a6-b82f-433fc7195e17'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.970115 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5901d59848f45d4bbbadbff624ea29a', 'request_id': 'req-9a292532-f032-45a6-b82f-433fc7195e17'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.978882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5901d59848f45d4bbbadbff624ea29a', 'request_id': 'req-9a292532-f032-45a6-b82f-433fc7195e17'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.990754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] The token cache is being invalidate because project a5901d59848f45d4bbbadbff624ea29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:11.991061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.991719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-9a292532-f032-45a6-b82f-433fc7195e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.993264 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1377/2764] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:11 2022] DELETE /identity/v3/projects/a5901d59848f45d4bbbadbff624ea29a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:11.998055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366919b-b5c1-42f5-8b03-2012484e8dd6 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.998352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366919b-b5c1-42f5-8b03-2012484e8dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.998639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366919b-b5c1-42f5-8b03-2012484e8dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.019439 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a366919b-b5c1-42f5-8b03-2012484e8dd6 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:12.020804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a366919b-b5c1-42f5-8b03-2012484e8dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.028794 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce25693-609e-4cd2-899a-7afceff3ed65 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.029885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1378/2765] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:12.033886 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.034562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.034921 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.035257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.036041 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.038220 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.038448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.038591 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.038954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.060704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:12.061992 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.062893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1388/2766] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:12.066736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.068600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.070167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1379/2767] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:12.074042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.081006 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a88d99-0944-476f-b71f-2747d67ed7d0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.083009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1a6c5-6479-4919-83c6-2e6845491319 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.085592 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.086192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.086471 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.086742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.087380 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.087938 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.088443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.088728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.089004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.089103 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.089750 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:12.090066 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.091638 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.099617 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1389/2768] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:12.103737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.108164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1380/2769] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:07:12.109044 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.113407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a896d-4583-4b30-b3de-4652ba157c4e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.118664 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.119159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9c48-8407-4a7d-a743-50bbb5ae63c8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.119271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.119546 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.119825 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.120467 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.122299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.123854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.124446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.124730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.125009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.125668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.127563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.127712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.135572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1390/2770] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:12.139154 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.151706 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecaf91-8b3c-48f1-b0ac-e5c580af6a9d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.156652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368eb67-517d-4429-969e-4426b2c7c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.157313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1381/2771] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:12.157442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368eb67-517d-4429-969e-4426b2c7c3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.157616 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368eb67-517d-4429-969e-4426b2c7c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.157897 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368eb67-517d-4429-969e-4426b2c7c3a6 admin admin] PATH_INFO: `/v3/projects/fd19e98f816d4f20aa6d7b05a871829f/users/3de20c213a5846c19f9a2199d60084b5/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.161113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.175102 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368eb67-517d-4429-969e-4426b2c7c3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd19e98f816d4f20aa6d7b05a871829f, user_id=3de20c213a5846c19f9a2199d60084b5, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.178156 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368eb67-517d-4429-969e-4426b2c7c3a6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.191615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1391/2772] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:12 2022] PUT /identity/v3/projects/fd19e98f816d4f20aa6d7b05a871829f/users/3de20c213a5846c19f9a2199d60084b5/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:12.201942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.202232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.202516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.227296 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48977-40f5-4eb4-9ab7-5e738f93f065 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.230318 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] MFA Rules not processed for user `3de20c213a5846c19f9a2199d60084b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:12.231559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.239023 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.292243 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.292856 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.293137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.293414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.294074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.295981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.305046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1392/2773] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:12.310237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.323449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1382/2774] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:12.323724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52523ce-ace2-41ca-bf86-c953f503c99c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.324559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.327952 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.328557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.328868 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.329148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.329807 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.331744 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.334356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f71ce-f8cf-4594-ba50-847eb2657c2b admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.338917 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.339507 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.339785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.340060 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.340797 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.342401 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.352727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1383/2775] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:12.357164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.365539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1393/2776] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:12.366736 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.368615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2512be6-185a-45d4-b37a-a1c735ff1214 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.426134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.426794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.427669 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.428096 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.428739 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.430366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.431705 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7521e58-3378-4671-bc9b-27207b0e3024 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.436359 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.436958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.437242 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.437545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.438217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.440115 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.440314 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1384/2777] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:12.445129 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.455960 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291757f-aa63-4cb4-ac7f-664e31c1e38f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.461209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.462012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.462290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.462565 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] PATH_INFO: `/v3/projects/f121588c65124d53a94d38ef163a64d2/users/143737a60b7b4ebba16ba3df84233ced/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.463406 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1394/2778] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:12.467217 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.478944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78a12d-0f16-48fe-aed2-53a045a7c216 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.479372 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f121588c65124d53a94d38ef163a64d2, user_id=143737a60b7b4ebba16ba3df84233ced, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.482208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.483605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff795fe-a183-4d86-aee6-5b2c7fd8c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.484231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff795fe-a183-4d86-aee6-5b2c7fd8c36a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.484513 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff795fe-a183-4d86-aee6-5b2c7fd8c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.484785 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff795fe-a183-4d86-aee6-5b2c7fd8c36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.485428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff795fe-a183-4d86-aee6-5b2c7fd8c36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.487553 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff795fe-a183-4d86-aee6-5b2c7fd8c36a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.495929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1385/2779] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:12 2022] PUT /identity/v3/projects/f121588c65124d53a94d38ef163a64d2/users/143737a60b7b4ebba16ba3df84233ced/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:12.500263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1395/2780] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:12.504414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.507428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faada9a-2adc-480f-b814-299faad823bc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.507721 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faada9a-2adc-480f-b814-299faad823bc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.507997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faada9a-2adc-480f-b814-299faad823bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.535509 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6faada9a-2adc-480f-b814-299faad823bc None None] MFA Rules not processed for user `143737a60b7b4ebba16ba3df84233ced`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:12.536827 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6faada9a-2adc-480f-b814-299faad823bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.546110 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.570883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c67632-6474-4364-aead-a2a6c50e9262 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.576803 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.577532 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.577826 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.578105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] PATH_INFO: `/v3/projects/b76ca36e17494555a1cf6a7870418b70/users/a1feb26fd6534071b663fde3602d0e3c/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.593585 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.594783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76ca36e17494555a1cf6a7870418b70, user_id=a1feb26fd6534071b663fde3602d0e3c, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.597745 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.611632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1386/2781] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:12 2022] PUT /identity/v3/projects/b76ca36e17494555a1cf6a7870418b70/users/a1feb26fd6534071b663fde3602d0e3c/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:12.612507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1396/2782] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:12.616723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.625095 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faada9a-2adc-480f-b814-299faad823bc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.626537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6faada9a-2adc-480f-b814-299faad823bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.681811 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b952316-d594-4e97-b575-9cd99877fada admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.688447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.688648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faada9a-2adc-480f-b814-299faad823bc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.689170 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.689481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.689776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.690427 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.692075 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.702276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1387/2783] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:12.706381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.716105 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cbd57-faa9-43df-9c64-4d5d0f2d669d admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.721374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ae30-4101-4051-86e2-cea0abb1fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.722047 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ae30-4101-4051-86e2-cea0abb1fcc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.722329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ae30-4101-4051-86e2-cea0abb1fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.722611 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ae30-4101-4051-86e2-cea0abb1fcc2 admin admin] PATH_INFO: `/v3/projects/b76ca36e17494555a1cf6a7870418b70/users/a1feb26fd6534071b663fde3602d0e3c/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.725078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ae30-4101-4051-86e2-cea0abb1fcc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76ca36e17494555a1cf6a7870418b70, user_id=a1feb26fd6534071b663fde3602d0e3c, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.727740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ae30-4101-4051-86e2-cea0abb1fcc2 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.738365 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.738567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1388/2784] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:12 2022] PUT /identity/v3/projects/b76ca36e17494555a1cf6a7870418b70/users/a1feb26fd6534071b663fde3602d0e3c/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:12.739982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.740285 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.740547 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.749531 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.749829 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.750114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.782208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] MFA Rules not processed for user `a1feb26fd6534071b663fde3602d0e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:12.783416 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.802958 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.804586 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.813970 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1397/2785] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:12.855186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1389/2786] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:12.866923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.868444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.897344 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.931139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35c978-c4d3-4e51-9975-89f96eb47b00 service glance] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.980915 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.981558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.981842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.982114 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.983857 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.045178 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.046649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.056993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1398/2787] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:16.719080 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.728771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5abd9f-2341-4005-a62c-bc2f20125e79 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.732320 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.732921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.733201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.733514 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.789127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:16.792126 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.793967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.806583 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1390/2788] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:17.392983 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:17.402641 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83249-ad71-4ca4-9e25-cf3f7977ca32 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:17.406276 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e34901-7253-4f5e-baab-29ea21314940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:17.406881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e34901-7253-4f5e-baab-29ea21314940 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:17.407163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e34901-7253-4f5e-baab-29ea21314940 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:17.407436 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e34901-7253-4f5e-baab-29ea21314940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:17.416523 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e34901-7253-4f5e-baab-29ea21314940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:17.418036 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e34901-7253-4f5e-baab-29ea21314940 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:17.428680 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1399/2789] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:24.648230 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:24.725912 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e251b-6a88-4bc5-8a91-539e0e4e69ef service cinder] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:24.731208 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:24.731862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:24.732217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:24.732500 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] PATH_INFO: `/v3/users/33917147fa3a430c82475ed7ca836cfb` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:24.733677 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33917147fa3a430c82475ed7ca836cfb)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:24.735632 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:24.790369 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33917147fa3a430c82475ed7ca836cfb'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.801184 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33917147fa3a430c82475ed7ca836cfb'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.801837 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33917147fa3a430c82475ed7ca836cfb'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.810177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33917147fa3a430c82475ed7ca836cfb'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.823673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33917147fa3a430c82475ed7ca836cfb'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.830017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1391/2790] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:24 2022] DELETE /identity/v3/users/33917147fa3a430c82475ed7ca836cfb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:24.834516 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:24.835030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:24.835139 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:24.856390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:24.857626 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:24.936756 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1400/2791] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:24.947146 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:24.948626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:25.012078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bef70-ccf0-4aee-ac42-bb44f00e1e88 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:25.015637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:25.016242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:25.016523 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:25.016845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:25.027588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:25.029069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:25.039209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1392/2792] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:25.320085 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:25.322186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:25.334499 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4a754-44ca-436f-aff8-28ddb0cbbb11 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:25.339850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:25.340495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:25.340777 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:25.341056 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] PATH_INFO: `/v3/projects/fd3cd3613caa4eaa9a7ea3cc7bf3333a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:25.342660 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3cd3613caa4eaa9a7ea3cc7bf3333a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:25.344750 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:25.366619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:25.398439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3cd3613caa4eaa9a7ea3cc7bf3333a', 'request_id': 'req-a337ae1f-1007-455b-905e-6b5bf2a3a256'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.406148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3cd3613caa4eaa9a7ea3cc7bf3333a', 'request_id': 'req-a337ae1f-1007-455b-905e-6b5bf2a3a256'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.414263 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3cd3613caa4eaa9a7ea3cc7bf3333a', 'request_id': 'req-a337ae1f-1007-455b-905e-6b5bf2a3a256'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.424888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] The token cache is being invalidate because project fd3cd3613caa4eaa9a7ea3cc7bf3333a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:25.425180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.425835 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.427330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1401/2793] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:25 2022] DELETE /identity/v3/projects/fd3cd3613caa4eaa9a7ea3cc7bf3333a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:25.431747 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:25.441281 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526809-4c56-4d50-8804-03edec36df94 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:25.454081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:25.502935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:25.503568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:25.503845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:25.504116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] PATH_INFO: `/v3/users/3de20c213a5846c19f9a2199d60084b5` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:25.505205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de20c213a5846c19f9a2199d60084b5)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:25.507155 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:25.554031 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de20c213a5846c19f9a2199d60084b5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.562074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de20c213a5846c19f9a2199d60084b5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.562629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de20c213a5846c19f9a2199d60084b5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.569590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de20c213a5846c19f9a2199d60084b5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.582447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de20c213a5846c19f9a2199d60084b5'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.588828 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1393/2794] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:25 2022] DELETE /identity/v3/users/3de20c213a5846c19f9a2199d60084b5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:25.875993 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:25.877404 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:25.904965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337ae1f-1007-455b-905e-6b5bf2a3a256 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:25.972944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:25.973538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:25.973906 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:25.974123 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] PATH_INFO: `/v3/projects/fd19e98f816d4f20aa6d7b05a871829f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:25.975237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd19e98f816d4f20aa6d7b05a871829f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:25.977201 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:26.040980 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd19e98f816d4f20aa6d7b05a871829f', 'request_id': 'req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.047305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd19e98f816d4f20aa6d7b05a871829f', 'request_id': 'req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.055358 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd19e98f816d4f20aa6d7b05a871829f', 'request_id': 'req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.072648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] The token cache is being invalidate because project fd19e98f816d4f20aa6d7b05a871829f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:26.072942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.073645 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.075146 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1402/2795] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:25 2022] DELETE /identity/v3/projects/fd19e98f816d4f20aa6d7b05a871829f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:26.079209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:26.086149 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:26.099359 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03f8a-3f1a-4d55-8571-04219540f354 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:26.162719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:26.163380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.163657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.163927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] PATH_INFO: `/v3/users/a1feb26fd6534071b663fde3602d0e3c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.165436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1feb26fd6534071b663fde3602d0e3c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:26.167394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:26.209084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:26.220402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1feb26fd6534071b663fde3602d0e3c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.228269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1feb26fd6534071b663fde3602d0e3c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.228898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1feb26fd6534071b663fde3602d0e3c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.235950 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1feb26fd6534071b663fde3602d0e3c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.248308 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-c34894e0-1c9e-4740-a201-dc7cea807cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1feb26fd6534071b663fde3602d0e3c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.254262 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1394/2796] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:26 2022] DELETE /identity/v3/users/a1feb26fd6534071b663fde3602d0e3c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:26.539441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:26.540829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:26.553291 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7e14f-8a0d-4f57-8c09-879bb7e6d765 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:26.590182 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:26.621434 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:26.622033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.622336 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.622610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] PATH_INFO: `/v3/projects/b76ca36e17494555a1cf6a7870418b70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.623743 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b76ca36e17494555a1cf6a7870418b70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:26.625682 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:26.674755 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b76ca36e17494555a1cf6a7870418b70', 'request_id': 'req-c4b95d21-0209-4163-8740-e6761750c79a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.682237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b76ca36e17494555a1cf6a7870418b70', 'request_id': 'req-c4b95d21-0209-4163-8740-e6761750c79a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.692334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b76ca36e17494555a1cf6a7870418b70', 'request_id': 'req-c4b95d21-0209-4163-8740-e6761750c79a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.704731 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] The token cache is being invalidate because project b76ca36e17494555a1cf6a7870418b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:26.705019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.705711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c4b95d21-0209-4163-8740-e6761750c79a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.707236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1403/2797] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:26 2022] DELETE /identity/v3/projects/b76ca36e17494555a1cf6a7870418b70 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:26.712940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.713232 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.713544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.736515 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:26.737870 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:26.801285 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1395/2798] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:26.870180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.870584 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.870951 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.892232 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:26.894371 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:26.902844 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1404/2799] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:26.907540 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:26.918546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf6dd-dc53-452c-8bfa-0c92eab9ea68 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:26.923925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:26.924795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.924976 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.925342 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.926210 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:26.926593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:26.928407 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:26.947774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1396/2800] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:26.952212 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:26.964257 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14f93-7ff8-4a35-980f-ac8df152226a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:26.969979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:26.970602 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.970888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.971163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.971799 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:26.977775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.010752 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1405/2801] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:26 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:27.014724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.025883 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549fe9-20ca-42eb-8c86-1ab3f7c31ff8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:27.097339 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:27.098085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.098306 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.098637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.099283 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:27.101214 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.125716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1397/2802] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:27 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:27.129842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.147483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b4e69-697e-4142-9368-d45f94ccfcd2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:27.152973 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:27.153697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.154052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.154477 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.155203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:27.156883 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.168979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1406/2803] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:27.174206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.186616 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6e67-e0c2-498c-9e72-66a780c723ab admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:27.193702 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3eed5b-e9d0-4feb-a382-6a6ba9e5e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:27.194488 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eed5b-e9d0-4feb-a382-6a6ba9e5e28d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.194778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eed5b-e9d0-4feb-a382-6a6ba9e5e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.195329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eed5b-e9d0-4feb-a382-6a6ba9e5e28d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.196207 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eed5b-e9d0-4feb-a382-6a6ba9e5e28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:27.198381 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eed5b-e9d0-4feb-a382-6a6ba9e5e28d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.213516 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1398/2804] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:27.216987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.228522 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956eeb7-b86f-4579-8f3e-449a9fc0a785 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:27.233343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:27.234098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.234503 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.234788 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] PATH_INFO: `/v3/projects/324a011120644fa0ac6fb05d2a5e0c7b/users/22bb8073953b4291b691db72db442718/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.253370 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=324a011120644fa0ac6fb05d2a5e0c7b, user_id=22bb8073953b4291b691db72db442718, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:27.256033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.270923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1407/2805] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:27 2022] PUT /identity/v3/projects/324a011120644fa0ac6fb05d2a5e0c7b/users/22bb8073953b4291b691db72db442718/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:27.284109 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.284399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.284806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.333388 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] MFA Rules not processed for user `22bb8073953b4291b691db72db442718`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:27.334725 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:27.346271 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:27.410715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1399/2806] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:27.422354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.424091 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:27.488533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46394-dd14-4b89-bf5d-60c4154ccda7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:27.492867 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d53c9-bb51-47b8-98cb-649c0f10981c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:27.493402 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d53c9-bb51-47b8-98cb-649c0f10981c service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.493712 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d53c9-bb51-47b8-98cb-649c0f10981c service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.493986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d53c9-bb51-47b8-98cb-649c0f10981c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.502447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d53c9-bb51-47b8-98cb-649c0f10981c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:27.503908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d53c9-bb51-47b8-98cb-649c0f10981c service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.513181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1408/2807] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:29.137429 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.139136 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.219205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffe792-13e9-41c5-b1a4-c0e7817c863b None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.223878 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.224518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.224804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.225083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] PATH_INFO: `/v3/users/563b7749ad0d4417a2f73b178105dd8a` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.226245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563b7749ad0d4417a2f73b178105dd8a)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.237746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.301160 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563b7749ad0d4417a2f73b178105dd8a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.309733 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563b7749ad0d4417a2f73b178105dd8a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.310366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563b7749ad0d4417a2f73b178105dd8a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.317522 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563b7749ad0d4417a2f73b178105dd8a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.331042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563b7749ad0d4417a2f73b178105dd8a'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.337651 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1400/2808] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:29 2022] DELETE /identity/v3/users/563b7749ad0d4417a2f73b178105dd8a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:29.342711 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.343186 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.343299 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.350314 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:29.365414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:29.366644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.426995 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1409/2809] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:29.436763 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.438205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.475704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:29.500448 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5ac1-278a-453d-b22c-d30b078fa7e3 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.504089 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64216cb-cb47-4bb1-b368-2c8c025bed93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.504723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64216cb-cb47-4bb1-b368-2c8c025bed93 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.505006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64216cb-cb47-4bb1-b368-2c8c025bed93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.505278 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64216cb-cb47-4bb1-b368-2c8c025bed93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.514586 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64216cb-cb47-4bb1-b368-2c8c025bed93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.516017 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64216cb-cb47-4bb1-b368-2c8c025bed93 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.526313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1401/2810] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:29.797172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.798697 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.811179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075846f4-060a-4d49-85d8-09db328e7432 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.815984 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.816686 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.816965 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.817233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] PATH_INFO: `/v3/projects/c8c0e351b22740849732bb995f4878b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.818374 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c0e351b22740849732bb995f4878b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.820769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.867389 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c0e351b22740849732bb995f4878b8', 'request_id': 'req-ffc763e7-2c19-401e-bcad-816d98fe2917'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.874087 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c0e351b22740849732bb995f4878b8', 'request_id': 'req-ffc763e7-2c19-401e-bcad-816d98fe2917'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.881725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c0e351b22740849732bb995f4878b8', 'request_id': 'req-ffc763e7-2c19-401e-bcad-816d98fe2917'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.892990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] The token cache is being invalidate because project c8c0e351b22740849732bb995f4878b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:29.893678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.894319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ffc763e7-2c19-401e-bcad-816d98fe2917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.895812 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1410/2811] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:29 2022] DELETE /identity/v3/projects/c8c0e351b22740849732bb995f4878b8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:29.901679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.901968 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.902244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.921205 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:29.922430 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.979967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1402/2812] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:29.988797 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.989080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.989351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.010124 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.011511 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.019649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1411/2813] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.024537 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.034267 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1086f5-7e8b-4ba9-95e7-3ba618e8a2fa None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.038723 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.040092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.040370 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.040642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.041325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:30.041717 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.043301 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.055181 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1403/2814] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:30.059386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.068442 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d5346-d802-4038-876e-7a426ddfc12c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.073304 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.073974 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.074250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.074518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.075148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.077313 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.106002 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1412/2815] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:30.110318 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.121067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cd9f2-491f-4d9b-adf1-cfa37188bd04 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.177286 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.178001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.178201 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.178481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.179116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.181029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.202141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1404/2816] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:30.206729 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.218450 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a20e-fcba-41c4-8c47-ca15cc2fe93c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.223079 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.223696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.223956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.224225 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.224890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.227086 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.237019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1413/2817] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:30.240933 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.251313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e112e-6696-4d37-bda3-2cff6a89952a admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.255860 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144337e8-f317-424d-840b-e0c8e4570f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.256451 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144337e8-f317-424d-840b-e0c8e4570f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.256729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144337e8-f317-424d-840b-e0c8e4570f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.256998 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144337e8-f317-424d-840b-e0c8e4570f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.258030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144337e8-f317-424d-840b-e0c8e4570f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.259737 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144337e8-f317-424d-840b-e0c8e4570f5d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.270503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1405/2818] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:30.275473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.285243 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eadbd-dc76-434c-971b-6e1b2be649a3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.290369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.291011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.291289 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.291562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] PATH_INFO: `/v3/projects/9c7415c9258c43d9a143dc9a3670fa4f/users/364c4a2b208a4fc5ba39a7d62dc63354/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.306904 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7415c9258c43d9a143dc9a3670fa4f, user_id=364c4a2b208a4fc5ba39a7d62dc63354, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.309610 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.323519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1414/2819] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:30 2022] PUT /identity/v3/projects/9c7415c9258c43d9a143dc9a3670fa4f/users/364c4a2b208a4fc5ba39a7d62dc63354/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:30.328806 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6392bc4-d275-4845-bad6-7e0f330d8c0e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.329164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6392bc4-d275-4845-bad6-7e0f330d8c0e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.329446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6392bc4-d275-4845-bad6-7e0f330d8c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.349322 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-e6392bc4-d275-4845-bad6-7e0f330d8c0e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.350555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-e6392bc4-d275-4845-bad6-7e0f330d8c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.387554 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:30.416534 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1406/2820] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:30.426133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.427826 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.453679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:30.485177 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af95c0-baea-44d2-8a27-701f3e447218 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.488769 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.489368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.489674 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.489949 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.499130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.500539 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.510215 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1415/2821] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:34.872323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:34.872850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:34.872850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:34.957352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] MFA Rules not processed for user `364c4a2b208a4fc5ba39a7d62dc63354`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:34.958862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:35.087936 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1407/2822] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:35.092763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.103150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76245-a4cb-412d-96a8-9d6c55956f0c service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.110105 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.110728 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.111014 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.111288 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.111928 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.113881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.145025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1416/2823] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:35 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:35.149562 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.160957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299d6f-a6a8-4684-8ce1-5db6904c53c7 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.234567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.235300 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.235673 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.236081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.236888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.239035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.272641 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1408/2824] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:35 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:35.276475 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.287414 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d181c2-3b01-4da1-9a99-db26ed798988 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.292620 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.293229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.293734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.294007 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.294647 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.296528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.312933 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1417/2825] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:35.319687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.332244 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22494ffc-a6de-4c2d-b569-89652877b022 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.340288 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.340974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.341257 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.341561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] PATH_INFO: `/v3/projects/a800991993264f82a5a9bb1494e58a1c/users/a33e9edb4f814e55adb445c943820e15/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.359546 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a800991993264f82a5a9bb1494e58a1c, user_id=a33e9edb4f814e55adb445c943820e15, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.362408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.379740 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1409/2826] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:35 2022] PUT /identity/v3/projects/a800991993264f82a5a9bb1494e58a1c/users/a33e9edb4f814e55adb445c943820e15/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:35.385898 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.407016 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ad76c-9c65-4a2c-889a-4844dcdbcead admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.471487 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5a0fb-b1a0-46db-b6ac-0c6619db33be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.472203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a0fb-b1a0-46db-b6ac-0c6619db33be admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.472552 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a0fb-b1a0-46db-b6ac-0c6619db33be admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.472894 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a0fb-b1a0-46db-b6ac-0c6619db33be admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.473649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5a0fb-b1a0-46db-b6ac-0c6619db33be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.475381 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5a0fb-b1a0-46db-b6ac-0c6619db33be admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.493120 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1418/2827] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:35.497112 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.508436 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134cc7-8b87-4933-9aaa-ebf71657592f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.513116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b74633-4857-4f94-96de-54ff67f81f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.513812 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b74633-4857-4f94-96de-54ff67f81f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.514224 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b74633-4857-4f94-96de-54ff67f81f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.514371 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b74633-4857-4f94-96de-54ff67f81f85 admin admin] PATH_INFO: `/v3/projects/a800991993264f82a5a9bb1494e58a1c/users/a33e9edb4f814e55adb445c943820e15/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.516343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b74633-4857-4f94-96de-54ff67f81f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a800991993264f82a5a9bb1494e58a1c, user_id=a33e9edb4f814e55adb445c943820e15, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.519325 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b74633-4857-4f94-96de-54ff67f81f85 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.530268 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1410/2828] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:35 2022] PUT /identity/v3/projects/a800991993264f82a5a9bb1494e58a1c/users/a33e9edb4f814e55adb445c943820e15/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:37.352761 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:37.353229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:37.353351 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:37.373948 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] MFA Rules not processed for user `b86d6ec3c196460c9c4e2fa4713fb2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:37.375421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:37.448447 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1419/2829] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:40.912552 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10e1d-36d6-40b2-a176-afd43d527adc None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:40.913518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10e1d-36d6-40b2-a176-afd43d527adc None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:40.913988 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10e1d-36d6-40b2-a176-afd43d527adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:40.947343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-39c10e1d-36d6-40b2-a176-afd43d527adc None None] MFA Rules not processed for user `a33e9edb4f814e55adb445c943820e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:40.948676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-39c10e1d-36d6-40b2-a176-afd43d527adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:40.964859 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:41.054394 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1411/2830] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:41.067361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:41.070138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:41.151681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:41.193390 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1498bff-4c2f-4dc9-b6d0-ac855b6612e6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:41.199375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9a524cfaa141409ff24c149b698be8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:41.200251 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:41.200533 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:41.200881 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:41.281495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:41.283562 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:41.295191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1420/2831] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:41.471588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:41.471876 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:41.472152 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:41.500447 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:41.502363 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:41.576866 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1412/2832] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:41.640678 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:41.642494 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:41.752046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d405ef-3f0d-4b09-915d-3472e8ebb338 service glance] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:41.761399 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269ee4-30e4-4db1-b771-c78b9e39ebe3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:41.762097 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269ee4-30e4-4db1-b771-c78b9e39ebe3 service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:41.762386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269ee4-30e4-4db1-b771-c78b9e39ebe3 service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:41.762663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269ee4-30e4-4db1-b771-c78b9e39ebe3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:41.774613 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269ee4-30e4-4db1-b771-c78b9e39ebe3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:41.776636 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269ee4-30e4-4db1-b771-c78b9e39ebe3 service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:41.789491 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1421/2833] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:07:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:07:43.832113 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:43.837559 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:43.927853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729ad25-8a12-4a73-93f8-db61a03d0b50 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:43.934380 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:43.935425 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:43.935962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:43.936392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] PATH_INFO: `/v3/users/22bb8073953b4291b691db72db442718` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:43.937746 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22bb8073953b4291b691db72db442718)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:43.940046 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.025672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22bb8073953b4291b691db72db442718'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:44.038051 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22bb8073953b4291b691db72db442718'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:44.038810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22bb8073953b4291b691db72db442718'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:44.040399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:44.055957 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22bb8073953b4291b691db72db442718'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:44.075618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22bb8073953b4291b691db72db442718'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:44.087153 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1413/2834] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:43 2022] DELETE /identity/v3/users/22bb8073953b4291b691db72db442718 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:44.089099 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.089644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.089784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.112528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:44.114271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:44.207250 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1422/2835] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:44.218496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:44.220190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:44.363836 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d42402-86ea-4da6-9256-cb01240225e8 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.381486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7579b-b292-4940-9fb8-159de4fb12d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.382692 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7579b-b292-4940-9fb8-159de4fb12d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.383186 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7579b-b292-4940-9fb8-159de4fb12d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.383670 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7579b-b292-4940-9fb8-159de4fb12d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.398814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7579b-b292-4940-9fb8-159de4fb12d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.400547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7579b-b292-4940-9fb8-159de4fb12d9 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.413901 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1414/2836] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:45.371022 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:45.373057 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:45.397373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a8246-3398-4792-bfd8-24371b3ffeba None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:45.406699 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:45.407651 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.408112 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.408681 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] PATH_INFO: `/v3/projects/324a011120644fa0ac6fb05d2a5e0c7b` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.415035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=324a011120644fa0ac6fb05d2a5e0c7b)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:45.417080 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:45.465908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:45.578998 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '324a011120644fa0ac6fb05d2a5e0c7b', 'request_id': 'req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.593172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '324a011120644fa0ac6fb05d2a5e0c7b', 'request_id': 'req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.603072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '324a011120644fa0ac6fb05d2a5e0c7b', 'request_id': 'req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.623518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] The token cache is being invalidate because project 324a011120644fa0ac6fb05d2a5e0c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:45.624158 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.624935 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7e5cf44a-e5c7-4729-9a33-8337a0eec4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.626927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1423/2837] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:45 2022] DELETE /identity/v3/projects/324a011120644fa0ac6fb05d2a5e0c7b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:45.631727 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.633628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.634124 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.658874 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:45.660496 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:45.747882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1415/2838] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:45.755918 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.756233 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.756519 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.786643 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:45.787833 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:45.802203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1424/2839] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:45.807854 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:45.828930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98211-8267-4e96-8f92-bae80b57739e None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:45.834313 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:45.834970 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.835786 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.836735 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.837513 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:45.837833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:45.839535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:45.863890 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1416/2840] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:45.869271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:45.904672 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a0e4-c30b-48ae-8055-2965e8892a0a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:45.914862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:45.915515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.915800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.916271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.916962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:45.919398 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:45.991986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1425/2841] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:45 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:45.999817 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:46.012095 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66322d-56dd-4ebe-86d9-fe86d3427f95 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:46.120476 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:46.121217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:46.121622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:46.122081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] PATH_INFO: `/v3/users` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:46.123058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:46.125780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:46.138354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:46.162820 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1417/2842] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:45 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:46.167180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:46.182229 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296ef40-96fc-4aa5-b191-8270c537e0cd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:46.187373 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:46.188142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:46.188490 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:46.188961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:46.189723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:46.193679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:46.209572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1426/2843] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:46.210706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:46.228135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58bd32-9782-4923-a380-1d5cc9fd582e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:46.238024 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f866392-ecc4-4ce5-a2a3-7d9b9159a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:46.238637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f866392-ecc4-4ce5-a2a3-7d9b9159a14d admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:46.238923 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f866392-ecc4-4ce5-a2a3-7d9b9159a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:46.239204 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f866392-ecc4-4ce5-a2a3-7d9b9159a14d admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:46.240721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f866392-ecc4-4ce5-a2a3-7d9b9159a14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:46.245143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f866392-ecc4-4ce5-a2a3-7d9b9159a14d admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:46.267222 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1418/2844] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:46.273150 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:46.303333 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6bac-f5c0-425d-a384-7ad4326a3f5d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:46.322075 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:46.323334 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:46.324046 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:46.324501 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] PATH_INFO: `/v3/projects/f6d36838a5324f2b8b67dffe358ae992/users/8bf31ad50a9a44b79bdacafab3b0c6c4/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:46.360735 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d36838a5324f2b8b67dffe358ae992, user_id=8bf31ad50a9a44b79bdacafab3b0c6c4, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:46.371722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:46.409528 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1427/2845] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:46 2022] PUT /identity/v3/projects/f6d36838a5324f2b8b67dffe358ae992/users/8bf31ad50a9a44b79bdacafab3b0c6c4/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:46.414078 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407064e-8070-46c3-8179-17e5cc0cc842 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:46.414421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407064e-8070-46c3-8179-17e5cc0cc842 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:46.414643 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407064e-8070-46c3-8179-17e5cc0cc842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:46.478183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-a407064e-8070-46c3-8179-17e5cc0cc842 None None] MFA Rules not processed for user `8bf31ad50a9a44b79bdacafab3b0c6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:46.479831 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-a407064e-8070-46c3-8179-17e5cc0cc842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:46.570143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1419/2846] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:46.581393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:46.583710 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:46.654234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3056eaf-9d80-4b83-9170-b85e57748e4c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:46.658302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6768d-5fbc-44f9-b709-ffbfc6086cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:46.658912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6768d-5fbc-44f9-b709-ffbfc6086cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:46.659190 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6768d-5fbc-44f9-b709-ffbfc6086cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:46.659461 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6768d-5fbc-44f9-b709-ffbfc6086cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:46.675054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6768d-5fbc-44f9-b709-ffbfc6086cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:46.677089 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6768d-5fbc-44f9-b709-ffbfc6086cc2 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:46.688968 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1428/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:52.583728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.584246 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.584336 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.613332 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] MFA Rules not processed for user `364c4a2b208a4fc5ba39a7d62dc63354`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:52.614553 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:52.684457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1420/2848] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:56.683466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:57.707979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:57.708884 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:57.709360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:57.764160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:57.765515 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:57.850331 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1429/2849] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:07:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:57.863062 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:57.864480 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:57.919019 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:57.947489 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32091809-5bb9-4905-8ac6-908857b2069d None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:57.952545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90305a4-3648-478a-8df6-f31427eb5a9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:57.953159 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90305a4-3648-478a-8df6-f31427eb5a9e service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:57.953443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90305a4-3648-478a-8df6-f31427eb5a9e service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:57.953757 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90305a4-3648-478a-8df6-f31427eb5a9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:57.962729 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90305a4-3648-478a-8df6-f31427eb5a9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:57.964321 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90305a4-3648-478a-8df6-f31427eb5a9e service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:57.972566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1421/2850] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:07:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:07:58.393942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:58.395346 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:58.491773 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508ce0d-61c6-4009-b6af-eee6f16d471d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:58.496521 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:58.497144 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:58.497421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:58.497723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] PATH_INFO: `/v3/users/143737a60b7b4ebba16ba3df84233ced` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:58.498860 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=143737a60b7b4ebba16ba3df84233ced)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:58.500764 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:58.546093 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143737a60b7b4ebba16ba3df84233ced'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:58.547909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:58.554572 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143737a60b7b4ebba16ba3df84233ced'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:58.555466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143737a60b7b4ebba16ba3df84233ced'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:58.556111 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143737a60b7b4ebba16ba3df84233ced'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:58.563151 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143737a60b7b4ebba16ba3df84233ced'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:58.580480 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1430/2851] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:58 2022] DELETE /identity/v3/users/143737a60b7b4ebba16ba3df84233ced => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:58.585891 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:58.586211 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:58.586492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:58.615061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:58.616317 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:58.677093 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1422/2852] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:58.687858 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:58.689742 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:58.747612 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53ba51-f0bd-481f-a651-ef98889024dd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:58.751356 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073257e-fda8-406e-b4d8-9d316e723ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:58.751964 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073257e-fda8-406e-b4d8-9d316e723ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:58.752245 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073257e-fda8-406e-b4d8-9d316e723ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:58.752518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073257e-fda8-406e-b4d8-9d316e723ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:58.763234 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073257e-fda8-406e-b4d8-9d316e723ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:58.764688 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073257e-fda8-406e-b4d8-9d316e723ca7 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:58.774801 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1431/2853] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:59.573544 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:59.575175 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:59.589927 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd0aed-dbf7-4882-943e-bb35e7a40fdb None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:59.594487 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:59.595116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:59.595395 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:59.596029 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] PATH_INFO: `/v3/projects/f121588c65124d53a94d38ef163a64d2` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:59.597107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f121588c65124d53a94d38ef163a64d2)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:59.599048 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:59.645861 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f121588c65124d53a94d38ef163a64d2', 'request_id': 'req-382d419f-763f-4de3-a301-763b1532a0e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:59.652979 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f121588c65124d53a94d38ef163a64d2', 'request_id': 'req-382d419f-763f-4de3-a301-763b1532a0e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:59.660610 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f121588c65124d53a94d38ef163a64d2', 'request_id': 'req-382d419f-763f-4de3-a301-763b1532a0e7'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:59.671672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] The token cache is being invalidate because project f121588c65124d53a94d38ef163a64d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:59.671961 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:59.672588 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-382d419f-763f-4de3-a301-763b1532a0e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:59.674120 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1423/2854] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:59 2022] DELETE /identity/v3/projects/f121588c65124d53a94d38ef163a64d2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:59.679209 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:59.679495 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:59.679770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:59.729271 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:59.730493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:59.787778 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1432/2855] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:59.795426 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:59.795751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:59.796033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:59.815904 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:59.817042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:59.825862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1424/2856] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:59.830019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:59.840303 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b036698-e4c1-4f3b-8278-3933bda56291 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:59.845029 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:59.845713 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:59.845931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:59.846203 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:59.846837 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:59.847135 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:59.848694 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:59.865270 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1433/2857] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:59.869668 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:59.878805 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058eaf-f011-4052-9433-a460bd511a9a None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:59.883940 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:59.884639 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:59.884918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:59.885190 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:59.885862 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:59.887751 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:59.916026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1425/2858] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:59 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:59.920226 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:59.930343 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42dbdd-4eec-4f76-a5be-ca148247855c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:59.961538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:59.992349 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:59.992962 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:59.993244 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:59.993566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:59.994202 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:59.996122 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:00.018439 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1434/2859] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:59 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:00.022844 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:00.033173 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0531fa-c814-4c52-b9f5-290f869bb01e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:00.038343 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:00.038882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:00.039164 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:00.039441 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:00.040084 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:00.041906 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:00.051911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1426/2860] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:00.056138 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:00.065977 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657612ec-1cee-458c-b243-ca2b72b94ae4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:00.071352 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab6626-cb02-4b5c-861e-ff59e3c748a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:00.071946 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab6626-cb02-4b5c-861e-ff59e3c748a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:00.072231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab6626-cb02-4b5c-861e-ff59e3c748a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:00.072512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab6626-cb02-4b5c-861e-ff59e3c748a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:00.073148 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab6626-cb02-4b5c-861e-ff59e3c748a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:00.074770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab6626-cb02-4b5c-861e-ff59e3c748a2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:00.085433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1435/2861] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:00.089493 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:00.099580 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100469ee-6ea1-4b70-80ca-8a15e32fcbcd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:00.104185 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:00.104821 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:00.105098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:00.105366 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] PATH_INFO: `/v3/projects/19a42d939d054ebc9b979610e499d57e/users/c8c61d6b587f464fac82ddb7d0a3d366/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:00.108392 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:00.122541 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a42d939d054ebc9b979610e499d57e, user_id=c8c61d6b587f464fac82ddb7d0a3d366, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:00.125221 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:00.138361 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1427/2862] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:00 2022] PUT /identity/v3/projects/19a42d939d054ebc9b979610e499d57e/users/c8c61d6b587f464fac82ddb7d0a3d366/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:00.150702 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:00.150992 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:00.151273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:00.179143 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] MFA Rules not processed for user `c8c61d6b587f464fac82ddb7d0a3d366`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:00.180361 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:00.243749 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1436/2863] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:00.255091 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:00.257202 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:00.326111 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d871d63-5e04-4a21-aa7e-9eafa02a997e admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:00.329804 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9966-6b8a-4716-b019-b33a25d0421b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07ba8c1958e4e2e82cdd4e490011593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:00.330445 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9966-6b8a-4716-b019-b33a25d0421b service cinder] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:00.330730 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9966-6b8a-4716-b019-b33a25d0421b service cinder] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:00.331012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9966-6b8a-4716-b019-b33a25d0421b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:00.340183 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b9966-6b8a-4716-b019-b33a25d0421b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:00.341667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b9966-6b8a-4716-b019-b33a25d0421b service cinder] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:00.351657 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1428/2864] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:06.813042 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:06.814707 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:06.887696 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7d842-c674-4455-b7a7-bef8da924307 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:06.892956 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:06.893718 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:06.894067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:06.894407 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] PATH_INFO: `/v3/users/8bf31ad50a9a44b79bdacafab3b0c6c4` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:06.895615 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf31ad50a9a44b79bdacafab3b0c6c4)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:06.898580 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:06.945944 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:06.952564 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf31ad50a9a44b79bdacafab3b0c6c4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.963100 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf31ad50a9a44b79bdacafab3b0c6c4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.963808 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf31ad50a9a44b79bdacafab3b0c6c4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.964673 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf31ad50a9a44b79bdacafab3b0c6c4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.976890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf31ad50a9a44b79bdacafab3b0c6c4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.005211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1437/2865] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:06 2022] DELETE /identity/v3/users/8bf31ad50a9a44b79bdacafab3b0c6c4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:07.010130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.010655 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.010974 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.035179 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:07.036548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.111964 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1429/2866] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:07.121394 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.122900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.186073 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4442d04-abb7-43f1-a60f-6caa8954dc04 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.190067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059406c5-1f4e-4720-a3e6-a7bc5b40dad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.190892 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059406c5-1f4e-4720-a3e6-a7bc5b40dad8 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.191363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059406c5-1f4e-4720-a3e6-a7bc5b40dad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.191802 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059406c5-1f4e-4720-a3e6-a7bc5b40dad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.205563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059406c5-1f4e-4720-a3e6-a7bc5b40dad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.205563 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059406c5-1f4e-4720-a3e6-a7bc5b40dad8 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.215931 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1438/2867] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:07.549780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.554452 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.582556 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f1c6e-83c8-4ae5-9634-e35a610c5a93 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.598250 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.598780 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.599067 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.599345 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] PATH_INFO: `/v3/projects/f6d36838a5324f2b8b67dffe358ae992` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.604684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d36838a5324f2b8b67dffe358ae992)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.606680 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.685231 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d36838a5324f2b8b67dffe358ae992', 'request_id': 'req-ceead1e7-d4bc-4955-9ae4-815949976cde'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.699597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d36838a5324f2b8b67dffe358ae992', 'request_id': 'req-ceead1e7-d4bc-4955-9ae4-815949976cde'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.721001 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d36838a5324f2b8b67dffe358ae992', 'request_id': 'req-ceead1e7-d4bc-4955-9ae4-815949976cde'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.747660 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] The token cache is being invalidate because project f6d36838a5324f2b8b67dffe358ae992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:07.748083 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.750216 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ceead1e7-d4bc-4955-9ae4-815949976cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.752687 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1430/2868] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:07 2022] DELETE /identity/v3/projects/f6d36838a5324f2b8b67dffe358ae992 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:07.761169 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.761537 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.761950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.794134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:07.795363 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.999783 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1439/2869] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:08.010324 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.010905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.010905 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.034486 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:08.036612 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:08.078215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1431/2870] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:08.082538 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.109246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70598037-4f39-44a1-ac44-c83a299cc6fe None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.120644 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.123800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.123800 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.124181 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.125140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:08.125597 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.129842 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.147441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1440/2871] 10.222.0.23 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:08:08.151492 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.159845 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:08.167618 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bfa04-9d11-43ee-bbe3-526e50aa9cee None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.174997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.177719 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.178548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.179057 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.179879 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.193795 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.233206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1432/2872] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:08 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:08.237758 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.250891 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51348ac-9319-4bef-b8fb-dddf7f20c05d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.301924 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:08.315882 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.316493 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.316776 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.317058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.317737 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.320213 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.346997 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1441/2873] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:08 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:08:08.348072 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.358630 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6f1d2-c2fa-4340-bd64-66bdde78c668 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.363384 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.363982 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.364265 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.364543 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.366234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.367760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.378548 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1433/2874] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:08.384061 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.394327 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5663970-125c-4869-b07a-3d984a998fc2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.399305 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72655f22-9c87-41e3-88c9-94334afe1514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.399900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72655f22-9c87-41e3-88c9-94334afe1514 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.400180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72655f22-9c87-41e3-88c9-94334afe1514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.400449 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72655f22-9c87-41e3-88c9-94334afe1514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.401081 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72655f22-9c87-41e3-88c9-94334afe1514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.402714 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72655f22-9c87-41e3-88c9-94334afe1514 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.413448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1442/2875] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:08.417771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.428176 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0385e8-1907-450a-bd30-da502f6d5016 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.432695 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.433338 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] REQUEST_METHOD: `PUT` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.433658 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.433937 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] PATH_INFO: `/v3/projects/dd7942cc29374e019b41f99337fecae2/users/9b0f3d7f72004c769efbb341276cb425/roles/3e2944846ed042f0bf3a6b134cb87695` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.448774 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7942cc29374e019b41f99337fecae2, user_id=9b0f3d7f72004c769efbb341276cb425, role_id=3e2944846ed042f0bf3a6b134cb87695)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.451504 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.465242 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1434/2876] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:08 2022] PUT /identity/v3/projects/dd7942cc29374e019b41f99337fecae2/users/9b0f3d7f72004c769efbb341276cb425/roles/3e2944846ed042f0bf3a6b134cb87695 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:08.470362 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160b0c9-6a78-4d4b-bb4e-00a2c389a802 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.470646 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160b0c9-6a78-4d4b-bb4e-00a2c389a802 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.470919 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160b0c9-6a78-4d4b-bb4e-00a2c389a802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.489912 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e160b0c9-6a78-4d4b-bb4e-00a2c389a802 None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:08.491630 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e160b0c9-6a78-4d4b-bb4e-00a2c389a802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:08.551803 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1443/2877] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:08.562209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.563918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:08.620628 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a29361-edf3-4680-99fe-321989b3cadd admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.624280 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.624887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.625168 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.625443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.635235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.636701 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.647623 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1435/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:13.663492 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:13.664019 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:13.664211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:13.700003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] MFA Rules not processed for user `9b0f3d7f72004c769efbb341276cb425`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:13.701605 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:13.786134 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1444/2879] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:13.790888 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:13.800930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90346441-2271-4b0f-a0aa-0fd016955b0d service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:13.807069 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:13.807732 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:13.808012 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:13.808289 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:13.808945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:13.810921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:13.847896 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1436/2880] 10.222.0.23 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:13 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:08:13.852211 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:13.890679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a7b4b-6e5c-4d70-95a9-7a137b80fdee None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:13.965266 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:13.973739 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:13.974648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:13.974979 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:13.975191 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:13.975910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:13.978030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:14.004893 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1445/2881] 10.222.0.23 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:13 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:14.009403 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:14.021769 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7477e52-7183-4e3f-8077-b43c86fd30ba admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:14.026814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:14.027457 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:14.027738 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:14.028013 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:14.028683 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:14.030354 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:14.042352 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1437/2882] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:14.046611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:14.057578 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9ffe6-2fa8-4d90-aec8-98f0e473c0e2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:14.062386 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:14.063053 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:14.063341 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:14.063614 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] PATH_INFO: `/v3/projects/0764210ebad142e9afab3b7644b49087/users/777c15861f714004ad6016dc84004308/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:14.079689 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0764210ebad142e9afab3b7644b49087, user_id=777c15861f714004ad6016dc84004308, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:14.082536 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:14.099034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1446/2883] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:14 2022] PUT /identity/v3/projects/0764210ebad142e9afab3b7644b49087/users/777c15861f714004ad6016dc84004308/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:14.103479 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:14.115903 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1abd-22ac-475d-89e1-e69a4a321e30 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:14.198379 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:14.199235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:14.199527 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:14.199809 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] PATH_INFO: `/v3/roles` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:14.200462 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:14.209802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:14.228266 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1438/2884] 10.222.0.23 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:14.229428 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:14.240258 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb56507-da5a-4756-a7f4-61a5890b221b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:14.253045 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f245b-cd33-4600-951d-27ebe7ed1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:14.253765 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f245b-cd33-4600-951d-27ebe7ed1d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:14.254051 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f245b-cd33-4600-951d-27ebe7ed1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:14.254329 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f245b-cd33-4600-951d-27ebe7ed1d82 admin admin] PATH_INFO: `/v3/projects/0764210ebad142e9afab3b7644b49087/users/777c15861f714004ad6016dc84004308/roles/8ab3a31c282a48a1b027ee33b288be76` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:14.256518 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f245b-cd33-4600-951d-27ebe7ed1d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0764210ebad142e9afab3b7644b49087, user_id=777c15861f714004ad6016dc84004308, role_id=8ab3a31c282a48a1b027ee33b288be76)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:14.259679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f245b-cd33-4600-951d-27ebe7ed1d82 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:14.272160 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1447/2885] 10.222.0.23 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:14 2022] PUT /identity/v3/projects/0764210ebad142e9afab3b7644b49087/users/777c15861f714004ad6016dc84004308/roles/8ab3a31c282a48a1b027ee33b288be76 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:19.460446 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:19.464151 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:19.507134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:19.711408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435044f5-0aac-43fd-a88a-d9ad68bf202c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:19.810721 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:19.811421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:19.811783 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:19.812134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:19.814568 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:19.829885 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:19.832008 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:19.850922 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1439/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:20.611020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:20.611600 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:20.611715 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:20.644986 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] MFA Rules not processed for user `777c15861f714004ad6016dc84004308`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:20.646869 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:20.723726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1448/2887] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:20.738033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:20.739621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:20.840410 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:20.843482 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae91d97-8ecf-45e2-b158-2a02ca8b6518 service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:20.850107 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:20.851303 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:20.851402 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:20.851622 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:20.918295 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:20.919898 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:20.929676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1440/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:30.720594 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=52,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:30.924945 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=26 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:36.210762 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:36.280098 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b8ba0-b672-4810-a614-0c0720dbe737 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:36.283910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:36.284679 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:36.285054 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:36.285332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:36.287241 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:36.371441 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:36.373050 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e62d-5a2b-4559-bfc9-94a1b939a7fd service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:36.384342 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1449/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:46.379388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=54 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:47.936038 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:47.937547 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:48.007494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfbf5d-e19f-4f7e-a14b-892ce6f1406f service nova] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:48.012594 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:48.013217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:48.013518 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:48.013794 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] PATH_INFO: `/v3/users/c8c61d6b587f464fac82ddb7d0a3d366` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:48.014918 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c61d6b587f464fac82ddb7d0a3d366)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:48.017116 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:48.074030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c61d6b587f464fac82ddb7d0a3d366'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.082698 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c61d6b587f464fac82ddb7d0a3d366'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.083328 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c61d6b587f464fac82ddb7d0a3d366'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.095375 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c61d6b587f464fac82ddb7d0a3d366'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.108404 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c61d6b587f464fac82ddb7d0a3d366'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.114269 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1441/2890] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:47 2022] DELETE /identity/v3/users/c8c61d6b587f464fac82ddb7d0a3d366 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:48.119278 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:48.119745 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:48.119862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:48.139354 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] MFA Rules not processed for user `5da7578088e04700b6983b884a95e6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:48.140559 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:48.200482 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1450/2891] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:48.210130 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:48.212646 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:48.283924 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c96d99-e74b-42de-bee9-0942e173b9ef admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:48.287566 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a569b6-f1ee-438b-9af2-2ecd34b54a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:48.288167 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a569b6-f1ee-438b-9af2-2ecd34b54a6d service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:48.288449 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a569b6-f1ee-438b-9af2-2ecd34b54a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:48.288724 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a569b6-f1ee-438b-9af2-2ecd34b54a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:48.298141 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a569b6-f1ee-438b-9af2-2ecd34b54a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:48.299597 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a569b6-f1ee-438b-9af2-2ecd34b54a6d service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:48.309800 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1442/2892] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:48.570914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:48.572512 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:48.584829 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840aa9c-f751-41b2-afa4-e6c2c868963a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:48.589662 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:48.590410 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:48.590723 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:48.590999 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] PATH_INFO: `/v3/projects/19a42d939d054ebc9b979610e499d57e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:48.592254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a42d939d054ebc9b979610e499d57e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:48.594253 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:48.643950 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a42d939d054ebc9b979610e499d57e', 'request_id': 'req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.650581 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a42d939d054ebc9b979610e499d57e', 'request_id': 'req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.658541 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a42d939d054ebc9b979610e499d57e', 'request_id': 'req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.669722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] The token cache is being invalidate because project 19a42d939d054ebc9b979610e499d57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:48.670011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.670796 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:48.672364 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1451/2893] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:48 2022] DELETE /identity/v3/projects/19a42d939d054ebc9b979610e499d57e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:49.207076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:49.207617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:49.207617 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:49.234629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] MFA Rules not processed for user `9b0f3d7f72004c769efbb341276cb425`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:49.236156 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:49.245058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:49.320637 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1443/2894] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:08:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:58.666601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:59.313765 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=20 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:04.932591 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:04.934326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:04.999433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd3295-cf1a-4e01-8bc2-9aa951c7da15 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:05.004008 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:05.004639 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:05.004917 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:05.005189 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] PATH_INFO: `/v3/users/b86d6ec3c196460c9c4e2fa4713fb2b7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.006320 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86d6ec3c196460c9c4e2fa4713fb2b7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:05.008855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:05.056180 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86d6ec3c196460c9c4e2fa4713fb2b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.063483 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86d6ec3c196460c9c4e2fa4713fb2b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.064197 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86d6ec3c196460c9c4e2fa4713fb2b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.064855 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86d6ec3c196460c9c4e2fa4713fb2b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.072303 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86d6ec3c196460c9c4e2fa4713fb2b7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.089425 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1452/2895] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:04 2022] DELETE /identity/v3/users/b86d6ec3c196460c9c4e2fa4713fb2b7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:05.340615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:05.354621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28c125-6aeb-4cfb-aa35-7c0aaebbc096 None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:05.421850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:05.422481 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:05.422760 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:05.423035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] PATH_INFO: `/v3/projects/15dd70cd3158499996e666026e293811` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.424135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15dd70cd3158499996e666026e293811)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:05.426098 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:05.472374 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15dd70cd3158499996e666026e293811', 'request_id': 'req-0e5564ab-2c72-4df9-99e9-883ffef7509c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.479600 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15dd70cd3158499996e666026e293811', 'request_id': 'req-0e5564ab-2c72-4df9-99e9-883ffef7509c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.487079 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15dd70cd3158499996e666026e293811', 'request_id': 'req-0e5564ab-2c72-4df9-99e9-883ffef7509c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.498134 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] The token cache is being invalidate because project 15dd70cd3158499996e666026e293811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:05.498421 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.499033 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.500499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1444/2896] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:05 2022] DELETE /identity/v3/projects/15dd70cd3158499996e666026e293811 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:07.902734 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.904133 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.962072 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cab80-5802-4425-9c55-a11ad72a83bf admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.993890 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:08.024034 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.024670 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.024955 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.025231 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.027125 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.091456 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.092817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c73b9-1f85-45ab-add4-56c3302c7a85 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.103601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1453/2897] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:15.495177 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:18.098396 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=34 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:52.270329 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:52.272323 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:52.284478 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5564ab-2c72-4df9-99e9-883ffef7509c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:52.288728 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:52.289319 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:52.289642 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:52.289929 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:52.291633 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:52.361434 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:52.362886 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:52.373889 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1445/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:00.105841 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:00.106360 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:00.106473 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:00.125959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] MFA Rules not processed for user `802cad51148347a189b9150bcf4fcf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:00.127142 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:00.181565 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1454/2899] 10.222.0.23 () {58 vars in 967 bytes} [Sat Sep 10 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:00.221704 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:00.223099 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:00.291399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e4a17-3fd2-48f1-b1cb-ed84da1813df service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:00.296362 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '233de9dbbb7a48159a1b0acdcd9880b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:00.296965 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:00.297245 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:00.297545 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:00.305955 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:00.307519 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:00.315593 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1446/2900] 10.222.0.23 () {60 vars in 1384 bytes} [Sat Sep 10 21:10:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:10:06.932192 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:06.933704 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:07.004272 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6015199-242c-4533-8efb-0d7e0722ede8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:07.010307 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:07.010987 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:07.011379 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:07.011692 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] PATH_INFO: `/v3/users/364c4a2b208a4fc5ba39a7d62dc63354` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:07.012923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=364c4a2b208a4fc5ba39a7d62dc63354)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:07.015052 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:07.068684 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364c4a2b208a4fc5ba39a7d62dc63354'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.076005 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364c4a2b208a4fc5ba39a7d62dc63354'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.076632 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364c4a2b208a4fc5ba39a7d62dc63354'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.077262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364c4a2b208a4fc5ba39a7d62dc63354'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.084780 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364c4a2b208a4fc5ba39a7d62dc63354'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.102236 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1455/2901] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:06 2022] DELETE /identity/v3/users/364c4a2b208a4fc5ba39a7d62dc63354 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:07.275206 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:07.286494 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7d6d6-c123-4232-bdfe-0659ed3e46a7 service swift] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:07.346139 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:07.346771 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:07.347055 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:07.347330 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] PATH_INFO: `/v3/projects/9c7415c9258c43d9a143dc9a3670fa4f` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:07.348931 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c7415c9258c43d9a143dc9a3670fa4f)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:07.350887 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:07.367629 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=100 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:07.396758 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c7415c9258c43d9a143dc9a3670fa4f', 'request_id': 'req-08b9d26e-faa3-4a2e-8121-737017fd8e1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.402930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c7415c9258c43d9a143dc9a3670fa4f', 'request_id': 'req-08b9d26e-faa3-4a2e-8121-737017fd8e1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.410248 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c7415c9258c43d9a143dc9a3670fa4f', 'request_id': 'req-08b9d26e-faa3-4a2e-8121-737017fd8e1c'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.421967 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] The token cache is being invalidate because project 9c7415c9258c43d9a143dc9a3670fa4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:10:07.422256 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.422882 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.424358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1447/2902] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:10:07 2022] DELETE /identity/v3/projects/9c7415c9258c43d9a143dc9a3670fa4f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:07.429184 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:07.430606 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:07.444631 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109d0ff-988f-485d-9305-8f842b3d7f5e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:07.464237 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:07.501069 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:07.501724 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:07.502003 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:07.502273 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] PATH_INFO: `/v3/users/a33e9edb4f814e55adb445c943820e15` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:07.503368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33e9edb4f814e55adb445c943820e15)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:07.505408 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:07.552448 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33e9edb4f814e55adb445c943820e15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.560279 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33e9edb4f814e55adb445c943820e15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.560907 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33e9edb4f814e55adb445c943820e15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.561545 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33e9edb4f814e55adb445c943820e15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.568558 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2fe42e13-d6d6-49da-9dc1-db1b92083008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33e9edb4f814e55adb445c943820e15'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.585393 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1456/2903] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:07 2022] DELETE /identity/v3/users/a33e9edb4f814e55adb445c943820e15 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:07.802555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:07.804569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:07.820004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9d26e-faa3-4a2e-8121-737017fd8e1c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:07.886555 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:07.887935 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:07.888215 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:07.888802 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] PATH_INFO: `/v3/projects/a800991993264f82a5a9bb1494e58a1c` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:07.889973 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a800991993264f82a5a9bb1494e58a1c)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:07.892209 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:07.940716 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a800991993264f82a5a9bb1494e58a1c', 'request_id': 'req-aa831b69-8fc0-4c93-9b88-9968f26fca0f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.947165 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a800991993264f82a5a9bb1494e58a1c', 'request_id': 'req-aa831b69-8fc0-4c93-9b88-9968f26fca0f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.955081 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a800991993264f82a5a9bb1494e58a1c', 'request_id': 'req-aa831b69-8fc0-4c93-9b88-9968f26fca0f'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.967128 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] The token cache is being invalidate because project a800991993264f82a5a9bb1494e58a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:10:07.967414 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.968076 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:07.969604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1448/2904] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:10:07 2022] DELETE /identity/v3/projects/a800991993264f82a5a9bb1494e58a1c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:17.581910 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:17.963952 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:10.782507 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:10.783062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:10.783062 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:10.802254 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] MFA Rules not processed for user `9b0f3d7f72004c769efbb341276cb425`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:10.804026 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:10.863457 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1457/2905] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:11.182006 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:11.183443 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:11.241563 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa831b69-8fc0-4c93-9b88-9968f26fca0f admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:11.245148 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:11.245790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:11.246074 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:11.246349 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:11.248087 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:11.257703 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:11.259135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:11.267584 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1449/2906] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:11.730763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:11.732262 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:11.806557 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38d38b-6afb-4957-9a5b-4b08e94f0802 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:11.811784 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:11.812624 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:11.812905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:11.813179 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:11.814947 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:11.827663 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:11.829246 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:11.840035 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1458/2907] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:14.013814 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:14.014962 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:14.027560 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827bc08-d5e5-45f7-aff1-81ada10a762d service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:14.039234 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:14.040309 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:14.040667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:14.041026 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:14.043118 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:14.055287 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:14.056981 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:14.068790 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1450/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:14.727566 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:14.737677 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:14.821791 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900678d7-8ad9-4a57-908e-18e759cf478c service placement] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:14.827011 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:14.827619 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:14.827900 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:14.828172 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:14.829908 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:14.843330 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:14.844923 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:14.855920 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1459/2909] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:24.062833 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=32 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:24.531777 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:24.546499 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94968-c2e4-4a31-9856-430bbe8bfd6c service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:24.553180 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:24.554438 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:24.554726 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:24.555003 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:24.556785 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:24.569577 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:24.571135 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cf633-fcd3-4ae6-9bf2-6e2cfb15e27f service nova] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:24.582147 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1451/2910] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:24.851340 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:27.878862 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:27.892726 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b079aa-9c57-4662-89ff-e65886a54b7e service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:27.900368 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:27.900982 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:27.901293 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:27.901611 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:27.903350 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:27.915990 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:27.917648 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:27.928961 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1460/2911] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:34.577356 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:37.923817 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:12.387396 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] REQUEST_METHOD: `POST` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.388408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.388408 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.407679 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.auth.core [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] MFA Rules not processed for user `9b0f3d7f72004c769efbb341276cb425`. Rule list: `[]` (Enabled: `True`). {{(pid=81440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:12.408858 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:12.415667 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1452/2912] 10.222.0.23 () {58 vars in 958 bytes} [Sat Sep 10 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:21.241419 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.253302 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66220870-6b75-4753-8433-46b62c8cf0f9 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.258725 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acfbe8c6a73d4b4992faba36c9881012', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.259309 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.259587 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.259859 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.261585 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:21.272554 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.274161 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.282976 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1461/2913] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:22.401792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=3 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:31.278967 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:10.084085 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:10.085517 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:10.100567 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311958a5-4fa6-4cb8-bfaf-4d5666e80c1c None None] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:10.105503 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:10.106127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:10.106406 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:10.106676 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:10.108360 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:10.120290 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:10.121911 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:10.132197 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1453/2914] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:20.127792 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=6 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:03.855021 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:03.864466 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a4c62-ad57-430b-843e-3fc8d0e9e11e service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:03.868058 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:03.868649 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:03.868927 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:03.869198 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:03.871027 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:03.885426 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:03.886888 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:03.899759 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1462/2915] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:13.896173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:15.911590 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.923684 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20a03-489f-4ef9-8157-1f904acd475f service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.927276 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.927871 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.928143 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.928417 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.930138 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:15.999561 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:16.001004 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:16.011650 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1454/2916] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:19.109375 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:19.124388 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f47d-c518-44b0-bd21-4dad4b2cf2bb service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:19.130863 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:19.133067 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:19.133568 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:19.134020 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:19.136116 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:19.149326 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:19.151127 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:19.162652 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1463/2917] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:22.336273 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.337851 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:22.351535 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee213d5d-2886-403b-a474-686387a36331 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.356119 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.356754 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.357035 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.357312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] PATH_INFO: `/v3/users/9b0f3d7f72004c769efbb341276cb425` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.358472 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0f3d7f72004c769efbb341276cb425)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.360415 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.408621 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0f3d7f72004c769efbb341276cb425'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.415997 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0f3d7f72004c769efbb341276cb425'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.416571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0f3d7f72004c769efbb341276cb425'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.424312 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0f3d7f72004c769efbb341276cb425'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.436217 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0f3d7f72004c769efbb341276cb425'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.442203 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1455/2918] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:22 2022] DELETE /identity/v3/users/9b0f3d7f72004c769efbb341276cb425 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:22.615698 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.626369 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd0c5-87e0-4ed8-9ce8-888204d8c091 service placement] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.684767 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.685312 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.685638 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.685914 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] PATH_INFO: `/v3/projects/dd7942cc29374e019b41f99337fecae2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.687030 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7942cc29374e019b41f99337fecae2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.688963 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.737332 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7942cc29374e019b41f99337fecae2', 'request_id': 'req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.743531 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7942cc29374e019b41f99337fecae2', 'request_id': 'req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.751497 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7942cc29374e019b41f99337fecae2', 'request_id': 'req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.763033 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] The token cache is being invalidate because project dd7942cc29374e019b41f99337fecae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:22.763319 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.763941 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.765422 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1464/2919] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:22 2022] DELETE /identity/v3/projects/dd7942cc29374e019b41f99337fecae2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:22.769925 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.771607 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:22.785569 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afec700-c9c8-41f1-a4c6-778e52603559 admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.840706 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.841334 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.841649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.841930 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] PATH_INFO: `/v3/users/777c15861f714004ad6016dc84004308` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.843061 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=777c15861f714004ad6016dc84004308)` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.845715 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.851133 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:22.892571 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777c15861f714004ad6016dc84004308'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.901776 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777c15861f714004ad6016dc84004308'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.902491 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777c15861f714004ad6016dc84004308'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.911101 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777c15861f714004ad6016dc84004308'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.922672 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.notifications [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777c15861f714004ad6016dc84004308'} {{(pid=81440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.928473 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1456/2920] 10.222.0.23 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:22 2022] DELETE /identity/v3/users/777c15861f714004ad6016dc84004308 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:23.104421 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.105909 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:23.119985 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6aa5bb-957a-4bbd-91ee-b86022e9f9d7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.178754 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da7578088e04700b6983b884a95e6ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '51e33b7d1c8d453f9d66204642c03a87', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.179384 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.179667 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.179942 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] PATH_INFO: `/v3/projects/0764210ebad142e9afab3b7644b49087` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.181048 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0764210ebad142e9afab3b7644b49087)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.182959 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.204850 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:23.232249 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0764210ebad142e9afab3b7644b49087', 'request_id': 'req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.238954 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0764210ebad142e9afab3b7644b49087', 'request_id': 'req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.246173 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0764210ebad142e9afab3b7644b49087', 'request_id': 'req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.256601 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] The token cache is being invalidate because project 0764210ebad142e9afab3b7644b49087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:23.256887 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.257549 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.259001 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1465/2921] 10.222.0.23 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:23 2022] DELETE /identity/v3/projects/0764210ebad142e9afab3b7644b49087 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:32.925058 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:33.253722 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:14.476358 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.477810 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.618219 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e9dd1-6aba-4fb5-aaa7-fe28e09f2f0c admin admin] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.677483 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df16503b8f2e4d9bb3cff4275a39d994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.678092 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.678373 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.678649 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.680376 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.693558 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.695137 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.705009 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1457/2922] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:15.419168 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.420953 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.504763 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec14e62-60a4-46ed-b2ca-ed243cf5d63d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.508290 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.508889 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.509163 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.509433 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.511170 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.601905 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.603335 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783da2-d5e3-4eaf-8038-64895cf9ac76 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.624130 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1466/2923] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:15.631475 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.640615 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebaffb-092e-47f4-a26b-8733288afb60 service placement] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.644778 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.645604 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.645921 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.646189 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.648333 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.fernet_utils [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] Could not find project: 2acae87b2c2e4b8bad6bdc4e5015bfe1.: keystone.exception.ProjectNotFound: Could not find project: 2acae87b2c2e4b8bad6bdc4e5015bfe1. Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 10 21:19:15.661865 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:19:15.663247 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:19:15.664549 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application with Lock( Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:19:15.665853 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:19:15.667042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2acae87b2c2e4b8bad6bdc4e5015bfe1. Sep 10 21:19:15.667042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: ERROR keystone.server.flask.application Sep 10 21:19:15.667042 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1458/2924] 10.222.0.23 () {60 vars in 1396 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 113 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:19:15.704770 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264b05-d4b1-4e4c-9495-c5c5a10316e8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.705140 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264b05-d4b1-4e4c-9495-c5c5a10316e8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.705444 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264b05-d4b1-4e4c-9495-c5c5a10316e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.763025 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d2264b05-d4b1-4e4c-9495-c5c5a10316e8 None None] MFA Rules not processed for user `acfbe8c6a73d4b4992faba36c9881012`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:15.764223 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d2264b05-d4b1-4e4c-9495-c5c5a10316e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.773658 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1467/2925] 10.222.0.23 () {58 vars in 1009 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:15.785030 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] Authenticating user token {{(pid=81440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.793731 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f162876-8dd7-4802-af09-5330395b0eb8 service neutron] Validating token access rules against request {{(pid=81440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.797235 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ae480-e2af-44ab-a075-f70d0a40c00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '934be918dc8c4fa59988e5bb22e56eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abec2d4d8c8e414092907eb50b7158b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.797850 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ae480-e2af-44ab-a075-f70d0a40c00b service neutron] REQUEST_METHOD: `GET` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.798127 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ae480-e2af-44ab-a075-f70d0a40c00b service neutron] SCRIPT_NAME: `/identity` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.798399 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ae480-e2af-44ab-a075-f70d0a40c00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.809114 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ae480-e2af-44ab-a075-f70d0a40c00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.810626 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ae480-e2af-44ab-a075-f70d0a40c00b service neutron] RBAC: Authorization granted {{(pid=81440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.821636 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: [pid: 81440|app: 0|req: 1459/2926] 10.222.0.23 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:25.760775 dv-cinder-851996-9-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:25.816875 dv-cinder-851996-9-iscsi devstack@keystone.service[81440]: DEBUG dbcounter [-] [81440] Writing DB stats keystone:SELECT=99 {{(pid=81440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}